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(54) Title: IMAGE PROCESSING APPARATUS AND METHOD 
(57) Abstract 

In a process for identifying the orientation of a line in an image, 
a histogram is formed of pixels of the line projected onto an axis. The 
axis is rotated and a histogram is formed of the pixels projected onto 
the rotated axis until the histogram includes charateristics indicating 
that the line is most closely perpendicular to the rotated axis. In a 
process of detecting a line on a road, an image is acquired of the 
road, and pixels of the image having characteristics corresponding 
to characteristics of the line are selected. A histogram is formed of 
the selected pixels projected onto an axis. The axis is rotated and a 
histogram formed of the selected pixels projected onto the rotated axis 
until the histogram includes characteristics indicative of a line. In a 
process of detecting a lane on a road having left and righ side lines, 
pixels of the image in a first area of the image at a first orientation 
and pixels in a second area of the image at a second orientation 
in the image are selected for selecting pixels associated with the 
lines. Histograms are formed projected on the first and second 
axes, respectively, and the axes are rotated until each histogram 
includes characteristics of a line. Also disclosed is a process of 
detecting a vehicle in an adjacent lane, systems for performing the 
aforementioned processes, systems for identifying an object and an 
input signal, and an interface kuween an image processing system 
and a controller. ^ 
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^^QmmOAPP MATxjs AND UFTHnn 



L Field of thf> Tn^n t ; rn 

The present invention relates general* to an image processing system, 
.«- .ore partly to an inteface betWee „ „ ^ ^ ^ 

J, o,,er, and to the use of a generic image processing system to detect me orientation 
of a „„e, to detect iines and lanes of a road, and to detect vehic.es o„ a road. 
2 - Descnptmn nf the Relate Art 

PCT/ E P98/053837T' y " OWned ^ APP ' iCa,i0n Seria ' N0S ' PC ™°,354 - 
PCT/EP98/05383 d,sclose a generic image-processing system that operates to localize 

objects ,„ reiative movement in an image and to determine the speed and direction of the 

objects ,n real-time. Each pixel of an image is smoothed using its own time constant A 

btnary value cording to the existence of a significant variation in the amplitude of 

he smoothed pixe, from the prior frame, and the amphtude 0 f the variation, are 

determmed, and the time constant for the pixel is updated. For each par.icu.ar pixel two 

matnces are fonned tha, indude a subset of the P «e,s spa,ia,.y related to the partis 

P.xel. The firs, matrix contains the binary vaiues of the subset of pixels. The second 

matnx contains the amplitude of the variation of the subset of pixels. In the firs, matrix, 

« » determined whether the pixeis a,o„g an oriented direction reiative to the particular 

p.xel have binary va.ues representative of significant variation, and, for such pixels i, is 

determmed in the second matrix whether the amp.itude of these pixels varies in a known 

manner mdicating movement in the oriented direction. ^domains that include 

lumtnance, hue, saturation, speed, oriented direction, time constant, and x and y position, 

a mstogram is formed of the vaiues in the first and second matrices faUing in user 

je.ec.ed combinations of such domai^. Using the histograms, i, is de.ermined whether 

.here . an area having the characteristics of the selected combinations of domains 
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It would be desirable to apply such a generic image processing system to 
detect various criteria useful in an automatic cruise control system, including detection of 
lines on a road, detection of lanes on a road, and detection of vehicular traffic in adjacent 
lanes. 

5 SUMMARY OF THE INVENTION 

The present invention is a process for identifying the orientation of a line 
in an input image, in which the line comprises a plurality of pixels. In the process, a 
histogram is formed of the pixels projected onto a first axis. The first axis is then rotated 
and a histogram is formed of the pixels projected onto the rotated first axis until the 
10 histogram includes characteristics indicating that the line is most closely perpendicular to 
the rotated first axis. The histogram characteristics preferably include R 
=NBPTS/RMAX, and the line is determined to be most closely perpendicular to the 
rotated first-axis at which R is a minimum. 

In a process of detecting a line on a road from a vehicle-mounted camera, 
15 an image is acquired of the road, with the image including pixels corresponding to the 
line. Pixels of the image having characteristics corresponding to characteristics of the 
line are selected and a histogram is formed of the selected pixels projected onto a first 
axis. The histogram is then analyzed to identify characteristics indicative of a line. The 
first axis is rotated and a histogram formed of the* selected pixels projected onto the 
20 rotated first axis until the histogram includes characteristics indicative of a line. The step 
of selecting pixels of the image having characteristics corresponding to characteristics of 
a line preferably involves selecting pixels from the group consisting of luminance, hue, 
saturation, direction, DP, CO and velocity. Alternatively, the step of selecting pixels 
having characteristics corresponding to characteristics of a line involves selecting pixels 
25 in a desired area of the image, possibly having a desired orientation in the image. If 
desired, the process of rotating the first axis is repeated and a histogram formed of the 
selected pixels until the histogram includes characteristics indicating that the line is most 
* closely perpendicular to the rotated first axis. 

The present invention is a process for identifying the orientation of a line 
30 in an input image, in which the line comprises a plurality of pixels. In the process, a 
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histogram is formed of the pUels projected onto' multiple „ Process ^ ^ 

5 the multiple axes at which R is a minimum. 

As non-limitative examples of multiple axes, 16 axes over .80- are used 
*e m,„ angle being bawm ^ - m ^ 1I 25 K fc aiK> possiWe (o ^ • 

axes wh,ch are perpendicular. Even if multiple axes are used, it is also possible t o further 
process by rotating those multiple axes until one of the multiple axes is most closely 
1" perpendicular to the line. y 

In a process of detecting a line on a road from a vehicle-mounted camera, 
an .mage ,s acquired of the road, with the image including pixels corresponding to the 
■ne. Pucels of tine image having characteristics corresponding to characteristics of the 

^ h,St ° Sram ™*~< >» <he characteristics indicative of a line 

Process until one of the histograms includes characteristics indicative of a line. The step 
o selecting pixels of the image having characteristics corresponding to characteristics of 
a l,ne preferably involves selecting pixels from the group consisting of luminance hue 
saturation, direction, DP, CO and velocity. Alternatively, the step of selecting pixels 
havmg characteristics corresponding to characteristics of a line involves selecting pixels 
■n a desired area of the image, possible having a desired orientation in the image If 
des,red, the process of rotating the multiple axes is repeated and forming histograms of 
he selected p,xe,s until one of the histograms inlcudes characteristics indicating tha, the 
line ,s most closely perpendicular to one of the multiple axes. 
' The process is done using parallel computation. 

to one embodiment, the line is a double line and the step of selecung 
pixeU of the image having characteristics corresponding to characteristics of the line 
solves selecting firs, pixeU in a firs, desired area of the^mage at a firs, desired 
onentation in the image for selecting pixels associated with the firs, line, and selecting 
P«e,s ,„ a second desired area of the image a. a second desired orientation in the image 
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for selecting pixels associated with the second line. In this embodiment, the step of 
forming a histogram of the selected pixels projected onto a first axis involves forming a 
first histogram of the selected first pixels projected onto a first axis and forming a second 
histogram of the selected second pixels projected onto the first axis, and the step of 
analyzing the histogram to identify characteristics indicative of a line includes analyzing 
each of the first and second histograms to identify characteristics indicative of a line. 
Finally, the step of rotating the first axis includes rotating the first axis and forming first 
and second histograms of the selected first and second pixels respectively projected onto 
the rotated first axis until each of the first and second histograms comprises 
characteristics indicative of a line. If desired, the step of rotating the first axis and 
forming first and second histogram may be repeated until each of the first and second 
histograms comprises characteristics indicative of a line and until at least one of the first 
and second histograms comprises characteristics indicating that the line associated with 
such histogram is most closely perpendicular to the rotated first axis. 
The double lines are preferably parallel to the other and each line may be a solid or 
broken line. 

In an alternative embodiment, the line is a parallel double line and the step 
of analyzing the histogram to identify characteristics indicative of a line includes 
analyzing the histogram to identify two peaks characteristic of a parallel double line. 

In one embodiment, in which the line is a broken line, the step of 
analyzing the histogram to identify characteristics indicative of a line involves time 
averaging the histogram over a succession of frames of the image. In an alternative 
embodiment in which the line is a broken line, the step of analyzing the histogram to 
identify characteristics indicative of a line involves analyzing the histogram over a 
succession of frames of the image to identify a periodic fluctuation in peaks of the 
histogram indicative of a broken line. In a further alternative embodiment in which the 
line is a broken line, the step of selecting pixels of the image having characteristics 
corresponding to characteristics of the line involves selecting fiiSt pixels in a first desired 
area of the image for selecting pixels associated with a first portion of the broken line, 
and selecting pixels in a second desired area of the image for selecting pixels associated 
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Tt l SeC °" d P0 " i0 " ° f br ° ke " fe -S— - fa, section . b 
embodiment the <!tPn of t.- a«,uon. in this 

»icm, ine step or forming a histogram of the wJp**«i • t 

■* and forming . second ^ a J^^ f* ~ - • 
line comprises analyzing the first an H . a u- cha ™tenst.cs indicative of a 

d 5 ,red area of the .mage a. a second desired orientation in the image are setaed for 
** a.ocia.ed * ,„ e second ,i„e. A firs, histogram^ ^£ 
. P«e,s proved onto a firs. a,s and a second his,ogra„, is formed of ,h 
« eced second p,«,s projected onto a second axis. Each of the firs, and second 
- grams . ana, V2 ed to identif, characteristics in each histogram indicative of aT 

™. and a firs „, slogram is formed of the foa ^ ^ 
*« «, and unfc, the second histogram inciudes characteristic, indicative of a hne the 
o „, rotated and a second histogram is formed of the second pixeis proj 2 

~ 7 1 ra,a,ing SeC °" d ■* - ' — "-gram ll 

repeated „„,,, each of the firs, and second histograms inciudes charac.eris.ics LZ 
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of a line and until at least one of the first and second histograms includes characteristics 
indicating that the line associated with such histogram is most closely perpendicular to 
the rotated first axis. 

In a process of detecting a vehicle in an adjacent lane from a camera 
mounted to a subject vehicle, an image of the adjacent lane is acquired. Pixels of the 
image having characteristics corresponding to characteristics of a vehicle are then 
selected. A histogram is formed of the selected pixels projected onto a first axis. Finally, 
the histogram is analyzed to detect characteristics indicative of a vehicle. - In such a 
process wherein the adjacent lane is defined by first and second side lines, the side lines 
may be identified using the process indicated above. Pixels of the image having 
characteristics corresponding to characteristics of a vehicle are then selected in an area 
bounded by the first and second side lines. If identifying taillights, the step of selecting 
pixels of the image having characteristics corresponding to characteristics of a vehicle 
may be accomplished by selecting pixels having a color or luminance characteristic of 
taillights. If desired, the histogram may be analyzed to separately detect each taillight. If 
identifying headlights, the step of selecting pixels of the image having characteristics 
corresponding to characteristics of a vehicle may be accomplished by selecting pixels 
having a color or luminance characteristic of headlights. The step of selecting pixels of 
the image having characteristics corresponding to characteristics of a vehicle may also 
involve selecting pixels moving in a direction parallel to a direction of the lane or 
selecting pixels moving in a direction generally parallel to one of the first or second side 
lines. The step of analyzing the histogram to detect characteristics indicative of a vehicle 
may comprise detecting a histogram having a minimum number of points. 

An apparatus for identifying the orientation of a line in an input image 
includes a histogram formation unit for forming a histogram of the pixels projected onto 
a first axis, and a controller for selectively rotating the first axis. The histogram 
formation unit forms a histogram of the pixels projected onto the rotated first axis, and 
the controller analyzes the histogram to determine when the histogram includes 
characteristics indicating that the line is most closely perpendicular to the rotated first 
axis. The histogram formation unit preferably includes a Hough transform unit for 



10 



15 



20 



25 



30 



WO 00/11609 

PCT/EP99/0042S 



Hough t^form on pb[e|s fM ^ ^ ^ ^ 
The hstogram formation uni, com P u,es R =NBPTS/RMAX 

the ..nemos, closely perpendicular to the rotated first axis. 

An apparatus for detecting a line on a road mcludes a vehicle-mounted 
-era aca,,n»g an image of , he roa , a ^ ^ , ^ £ 

fbrmmg ... hrstogram on pixels having se.ec.ed characteristics on a .elected axis The 
poller controls the histogram formation unit to se.ec, pixels of the image havi!g 
h-c ensucs corresponding to characteristics of a line and , form a histogram 
0J ec ed onto a firs, axis. The confer analyzes ,he his,o g ram ,o idel" 
charactenst.es mdicative of a .inc. The confer taher rotates the first axis Z 

-s. The controller analyzes the histogram of each rotated axis until the histogram 
compnses characteristics indicative of a line. The controUer preferably rotates me firs, 
£ «. I the confer determines that the histogram comprises characteristics indicating 
ha, the „„e ,s most closely perpendicular ,o me rotated first axis. The histogram ■ 
ormanon u„ preferably compu,es R =NBPTS/RMAX, and the controller deterrles 
he rota,, firs, axis a, which R is a minimum to identify «h 6 line most a J y 
P^P-feuUr ,o ,he rotated firs, axis. The seleced pixel characeristics are preferably 
selected from the group consisting of luminance, hue, saturation, direction, DP, CO and 

Ifdesired, the histogram formation uni, includes an area selection memory 
for selecung an area of an image for which ,o form a histogram. The Roller conj 
4. h„ogram formation unit ,o select pixels in a desired area of ,he image for detecting 
the hne. The his,ogram formation uni, may also include an angle selection memory for 
se.ect.ng an orientation angle for forming a histogram. The controller controls the 
histogram formation unit to select pixels in a desired area of the image and to form a 
hologram a, a desired Srientation angle for detecting the line. 

In one embodiment, the line is a double line and the controller controls 
-he histogram formation urn, , 0 seta pixels of ,he image having characeristics 
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corresponding to characteristics of a line in a first desired area of the image at a first 
desired orientation in the image for selecting pixels associated with a first line of the 
double line, and controls the histogram formation unit to select pixels in a second desired 
area of the image at a second desired orientation in the image for selecting pixels 
5 associated with the second line. The histogram formation unit forms a first histogram of 
the selected first pixels projected onto the first axis and forms a second histogram of the 
selected second pixels projected onto the first axis. The controller analyzes each of the 
first and second histograms to identify characteristics indicative of a line and rotates the 
first axis until each of the first and second histograms comprises characteristics indicative 
10 of a line. The controller may further rotate the first axis until each of the first and second 
histograms comprises characteristics indicative of a line and until at least one of the first 
and second histograms comprises characteristics indicating that the line associated with 
such histogram is most closely perpendicular to the rotated first axis. 

In an alternative embodiment in which the line is a parallel double line, the 
15 controller analyzes the histogram to identify two peaks in the histogram characteristic of 
a parallel double line. In an embodiment in which the line is a broken line, the controller 
time averages the histogram over a succession of frames of the image to identify 
characteristics indicative of a broken line. In an alternative embodiment in which the line 
is a broken line, the controller time analyzes the histogram over a succession of frames of 
20 the image to identify a periodic fluctuation in peaks of the histogram indicative of a 
broken line. In a still further alternative embodiment in which the line is a broken line, 
the controller controls the histogram formation unit to form a first histogram of first 
pixels in a first desired area of the image associated with a first portion of the broken 
line, and to form a second histogram of second pixels in a second desired area of the 
25 image associated with a second portion of the broken line adjacent to the first section, 
with each of the first and second histograms being projected onto the first axis. The 
controller analyzes the first and second histograms over a succession of frames of the 
image to iderfify a periodic movement of first pixels associated with the line from the 
first desired area to the second desired area. 
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fcaogram formation unit for seIecti „ g pixe , s fa ^ ^ ^ - J 

and for forming a histogram of the selected pixels. The controller con,ro,s the histogr^ 

fomung a htstogram of the selected pixels i. the firs, desired area projected olo . J 
ax.s for forming a firs, histogram „ f pixeIs ^ ^ 

controls .histogram formation uni, for selecting pixel s i„ . second desired area oft 
™*e and for forcing . second histogram of the selected pixels in the second de2 

z r e T r° a second * for forming a h ~ * ** -»<*- *■* - 

-n .me The controller analyzes each of the firs, and second histograms ,„ identify 
h,ac«er.s„cs ,„ each his,ogram indicate of a line. Fi n a„ y , the controller rotates the 
» u„„„he firs, histogram comprises characters indicative of a line, and rotates 
4. second ax,s unti, the second histogram comprises characteristics indicative of a line 
fdes,red, to*^^*^***^^^^^ 
firs, and second histograms comprises characeristics indicative of a line and until atleas. 
one of the firs, and second histograms comprises characteristics indicating that the iine 
assocated with such histogram is most closely perpendicular ,o the rotated firs, axis 

An appara,us for de.ec.ing a vehicle in an adjacem tane from , 
veh.Ce mcludes a camera mourned ,o ,he subject vehicle for acquiring an image of the 
adjacent ,ane, a histogram formation uni, for selecting pixels of me image and for 
ormmg a histogram of such images, and a controHer for controlling the histogram 
formanon uni, to selec, pixe.s having charac,eris,ics corresponding to characteristics of a 
veh,c.e and for analyzing the histogram of such pixels ,o de,ec, characteristics indicative 
of a vehicle. ,f the adjacent lane is defined hy first and second side lines, the side lines 
are detected as described above, and the controller controls the histogram formation unit 
■o select p,xe,s of the image having Caracas corresponding to characteristics of a 
vehtcle comprises in an area bounded by ,he side lines. If desired, the controller controls 
•he histogram formation uni, to select pixels i) having the color or luminance 
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characteristics of taillights, ii) having color or luminance characteristics of headlights, ii) 
moving in a direction parallel to a direction of the lane, or iv) moving in a direction 
generally parallel to one of the side lines. 

An apparatus for identifying an object in an input signal, in which the 
object includes pixels in one of a plurality of classes in one of a plurality of domains, 
includes: 

a classifier for each domain, the classifier classifying pixels within 
each domain in selected classes within the domain; 

a linear combination unit for each domain, the linear combination unit 
generating a validation signal for the domain, the validation signal selecting one or more 
of the plurality of domains for processing; 

a rotation unit for enabling selection of a histogram formation axis; 
a histogram formation unit for forming a histogram for pixels of the 
output signal within the classes selected by the classifier within each domain selected 
by the validation signal projected onto the histogram formation axis; and 

a controller for controlling the classifier, linear combination unit, rotation 
unit, and histogram formation unit for identifying the object. 

The rotation unit preferably enables selection of a first histogram 
formation axis and a second histogram formation axis, and the histogram formation unit 
is capable of forming a first histogram projected onto the first histogram formation axis, 
and of forming a second histogram projected onto the second histogram formation axis. 

An alternative apparatus for identifying an object in an input signal 
includes a classifier for each domain, a linear combination unit for each domain, an area 
selection unit for selecting an area of the image, a histogram formation unit, and a 
controller. 

The apparatus also preferably includes a rotation unit for enabling 
selection of a histogram formation axis, with the histogram formation unit forming a 
histogram for pixels of the output signal projected onto the histogram Irmation 
axis. 
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classifier . r A T ~ " *** * " ^ *- « 

class,fie, a 1 near comb,nau„„ uni, a masking unit for masking an ^ of * ^ ,„ 

5 comprises: iWerfaCe ^ taa8e ~" g «- - 3 «-» 

control 5 ,gnal S selected from the group consisting of 
.0 system, ' ^ " ^ * * "* P™—. 

selected »es,a^^ 

iv) signals for selecting an area of an image for processing by the image 
processing system; and = image 

output signals from the image processing system to the controller 
mcludrng stgnals resultant from processing the input signais selected from the group 
consisting of: & p 

i) signals containing information on histograms formed in the image 
processing system, and 

ii) signals containing histograms formed in the image processing system 
The domains are preferably selected from the group consisting of 

lununance, hue, saturation, CO. DP, direction, and velocity, and the signals staining 
nformauon on histograms formed in the image processing system are preferably selected 
from the group consisting of MTN, MAX, NBPTS, RMAX, POSRMAX. 
The apparatus is built using a single chip (MOS). 
The physical link of the interface is a standard automc*ve bus 1. is 
posstble to dynamically adapt in function 0 f results a. leas, one or more of the Mowing 
parameters: classification, areas, histograms. 
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It is possible to use an interface with a physical link as summarised. It is 
also possible to use the process and apparatus to determine curves, said curve being 
segmented into sensibly linear portions. 

BRIEF DESCRIPTION OF THE DRAWINGS 

Fig. 1 is a diagrammatic illustration of the system according to the 

invention. 

Fig. 2 is a block diagrafn of the temporal and spatial processing units of 

the invention. 

Fig. 3 is a block diagram of the temporal processing unit of the invention. 
Fig. 4 is a block diagram of the spatial processing unit of the invention. 
Fig. 5 is a diagram showing the processing of pixels in accordance with 

the invention. 

Fig. 6 illustrates the numerical values of the Freeman code used to 
determine movement direction in accordance with the invention. 

Fig. 7 illustrates nested matrices as processed by the temporal processing 

unit. 

Fig. 8 illustrates hexagonal matrices as processed by the temporal 
processing unit. 

Fig. 9 illustrates reverse-L matrices as processed by the temporal 
processing unit. 

Fig. 10 illustrates angular sector shaped matrices as processed by the 
temporal processing unit. 

Fig. 1 1 is a block diagram showing the relationship between the temporal 
and spatial processing units, and the histogram formation units. 

Fig. 12 is a block diagram showing the interrelationship between the 
various histogram formation units. 

Fig. 13 shows the formation of a two-dimensional histogram of a moving 
area from two one-dimensional histograms. % 

Fig. 14 is a block diagram of an individual histogram formation unit. 
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R«. 16 illuarates a one-dimensional histogram 

o^ionorat:::^ the use ° f " r- ° f the «— '° *« * 

Fig. 24 illustrates another embodiment of the invention. 

fig. 25 illustrates the use of the sv^m „p*i. • 

»e or tne system of the invention to detect curves 

The present invention discloses a number of applications for th, „ • 

r o rir " " aUl0nU " C ^ COntr01 ^ ^ ***» "«« on a 
de,ec„on of road ,ane S , and de(ection of vehjcu|ar traffic °" 

d.sc^.sanin.erfacefor^a.enericim^processingsys.em. 

The apparatus of the invention is similar to th>t a -u , ■ 
— - PCX Appfation « No , PCT/FR97/01354 2 £££ 
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which will be described herein for purposes of clarity. Referring to Figs. 1 and 10, the 
generic image processing system 22 includes a spatial and temporal processing unit 1 1 in 
combination with a histogram formation unit 22a. Spatial and temporal processing unit 
11 includes an input 12 that receives a digital video signal S originating from a video 
5 camera or other imaging device 13 which monitors a scene 13a. Imaging device 13. is 
preferably a conventional CMOS-type CCD camera, which for purposes of the presently- 
described automatic cruise-control application is preferably mounted on a vehicle facing 
the road in front of the vehicle. It will be appreciated that the system described may be 
used to detect other criteria useful for automatic operation of a vehicle, or other 
1 0 applications, and that the camera may be mounted on the vehicle in any desired fashion 
to detect the specific criteria of interest, or may otherwise be appropriately mounted for 
the desired application. It is also foreseen that any other appropriate sensor, e.g., 
ultrasound, 1R, Radar, etc., may be used as the imaging device. Imaging device 13 may 
have a direct digital output, or an analog output that is converted by an AJD convenor 
15 into digital signals. Imaging device 13 may also be integral with generic image 
processing system 22, if desired. 

While signal S may be a progressive signal, it is preferably composed of a 
succession of pairs of interlaced frames, TR, and TR', and TR 2 and TR' 2 , each consisting 
of a succession of horizontal scanned lines, e.g., 1,.„ luv-Ji.n in TR,, and z, in TR 2 . 
Each line consists of a succession of pixels or image-points PI, e.g., a,.,, a u and a,. 3 for 
line I,,; aln, and al,™ for line !,;„ ; al„ and a, 2 for line l„. Signal S(PI) represents 

signal S composed of pixels PI. 

S(PI) includes a frame synchronization signal (ST) at the beginning of 
each frame, a line synchronization signal (SL) at the beginning of each line, and a 
25 blanking signal (BL). Thus, S(PI) includes a succession frames, which are representative 
of the time domain, and within each frame, a series of lines and pixels, which are 
representative of the spatial domain. 

In the time dolnain, "successive frames" shall refer to successive frames of 
the same type (i.e., odd frames such as TR, or even frames such as TR 1 ,), and 
30 "successive pixels in the same position" shall denote successive values of the pixels (PI) 
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i« tt. same location in successive frames of*, same type, e.g., a ,., of , m ^ 
a™ auoflum the next corresponding frame TR 2 

Spatial and temporal processing unit 1, generates output, ZH and SR 14 
to a data bus 23 (Fig. 1 „. which are preferably digital sigMl , Complex signal ZH 
compnses a number of output signals generated by tbe system, preferably including 
s.gn,s ,nd,ca,,„g the existence and location of an area or objec in moBO „, an d £ 
speed V and the oriented direction of dis pl aceme„« DI of each pixel of the image Also 
preferably output from the system is input digital video signal S, which is delayed (SR) to 
make ,. synchronous with the output ZH for the frame, taking i„, 0 account the 
calculate time for the data in composite signal ZH (one frame). The delayed signal SR 
■s used , 0 display lhe image receiyed by ^ ]3 ^ a ^ te|evision ^ ■ 

wh.cn may also be used to display the information contained in composite signal ZH ' 
Composite signal ZH may also be transmitted to a separate processing assembly ,0a it, 
whtch further processing of the signal may be accomplished. 

Referring to Fig. 2, spatial and temporal processing unit 1 1 includes a first 
assembly Ha, which consists of a. temporal processing unit „ having an associated 
memory 16, a spatial processing unit 17 having a delay unit 18 and sequencing um, 19 
and a pixel Cock 20, which generates a Cock signa, HP. and which se^es as a clock for 
temporal processing unit ,5 and sequencing unit ,9. Clock pulses HP are generated by 
clock 20 at the pixel rate of the image, which is preferably ,3.5 MHZ. 

Fig. 3 shows the operation of temporal processing unit 15, the function of 
wmch „ to smooth the video signa! and generate a number of outputs that are utilized by 
spanal processing unit ,7. During processing, temporal processing unit 15 retrieves 
from memory ,6 the smoothed pixel values LI of the digital video signal from the 
«nmed,a.ely prior frame, and the values of a smoothing time constant CI for each pixel 
As used herein, LO and CO shall be used to denote the pixel values (L) and time 
constants (C) stored in memcy 16 from temporal processing unit 15, and LI and CI shall 
denote the pixel values (L) and«„e consents (C) respectively for such values retrieved 
from memory .6 for use by temporal processing unit ,5. Temporal processing unit ,5 
generates a binary output signal DP for each pixe!, which identifies whether the pixel has 
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undergone significant variation, and a digital signal CO, which represents the updated 

calculated value of time constant C. 

Referring to Fig. 3, temporal processing unit 15 includes a first block 15a 
which receives the pixels PI of input video signal S. For each pixel PI, the temporal 

5 processing unit retrieves from memory 16 a smoothed value LI of this pixel from the 
immediately preceding corresponding frame, which was calculated by temporal 
processing unit 15 during processing of the immediately prior frame and stored in 
memory 16 as LO. Temporal processing unit 15 calculates the absolute value AB of the 
difference between each pixel value PI and LI for the same pixel position (for example 

10 a t i, oflu in TRi and oflu in TR 2 : 

AB = |PI-LI I 

Temporal processing unit 15 is controlled by clock signal HP from clock 
20 in order to maintain synchronization with the incoming pixel stream. Test block 15b 
of temporal processing unit 15 receives signal AB and a threshold value SE. Threshold 
15 SE may be constant, but preferably varies based upon the pixel value PI, and more 
preferably varies with the pixel value so as to form a gamma correction. Known means 
of varying SE to form a gamma correction is represented by the optional block 15e 
shown in dashed lines. Test block 15b compares, on a pixel-by-pixel basis, digital s.gnals 
AB and SE in order to determine a binary signal DP. If AB exceeds threshold SE, wluch 
20 indicates that pixel value PI has undergone significant variation as compared to the 
smoothed value LI of the same pixel in the prior frame, DP is set to T for the pl xel 
under consideration. Otherwise, DP is set to "0" for such pixel. 

When DP = 1, the difference between the pixel value PI and smoothed 
value LI of the same pixel in the prior frame is considered too great, and temporal 
25 processing unit 15 attempts to reduce this difference in subsequent frames by reducmg 
the smoothing time constant C for that pixel. Conversely, if DP = 0, temporal processmg 
unit 15 attempts to increase this difference in subsequent frames by increasmg the 
smithing time constant C for that pixel. These adjustments to time constant C as a 
function of the value of DP are made by block 15c. If DP = 1, block 15c reduces the 
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time constant by a unit value 1 1 cn th** *u 

y a unit value U so that the new value of the time constant CO eouatafc, 
old value of the constant CI minus unit value U. 

CO=CI-U 

^ew va, U e of the liroe CQ equa]s , he o)d ^ ^ Mnarat a ^ 

co=ci+u 

unit .5b and e3Ch ** b, ° Ck bi "^ ^ DP from test 

,3b - - «— • CI from memory l6 , adjusts CI up M down 

i:r; c r a new ,ime cons,aM co " — - — * » * 

In a preferred embodiment, time constant C, is in ,he f orm 2 > where „ . 

u u p.. ' 5c adds one t0 P in tin ie constant 2" 

and thus simplifies the structure of block 15c. 

Firs, CO ^'"''""'"""^"^"—P^-P-Honofthesystem. 
First, CO must reman, within denned limit, ,„ . preferTed embodi CQ ^ 

beco me negative (C0 > o, and it mua not exceed a iimi, N (CO < N) which h 

In ; he ins,ance in wHch ci and co are in *• f °™ * *• ~ 

N is the maximum value for p. 

The upper limit N may be constant, but is preferably variable. An 

.olTr' ,5f indudes a resis,er ° r ** *° - ~* 

« vary N. The consequence of increasing N is to increase the sensitivity of the system 
° detectmg dispiacement of pUe,s, whereas reducing N improves delion of ^ 

d-*0 m order to reguiate the variation of LO as a of the ievel of P, i e N- 

! f e PI t u ' a ' io ; of wMch is done in block i5? wMch fa tws - -vi 

tne value of PI from video camera 13. 
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Finally, a calculation block 15d receives, for each pixel, the new time 
constant CO generated in block 15c, the pixel values PI of the incoming video signal S, 
and the smoothed pixel value LI of the pixel in the previous frame from memory 16. 
Calculation block 1 5d then calculates a new smoothed pixel value LO for the pixel as 
follows: 

LO=LI + (PI - LI)/CO 
If CO = 2 P , then 

LO=LI + (PI - LI)/2 po 

where "po", is the new value of p calculated in unit 15c and which replaces previous 
value of "pi" in memory 16. 

The purpose of the smoothing operation is to normalize variations in the 
value of each pixel PI of the incoming video signal for reducing the variation differences. 
For each pixel of the frame, tempgral processing unit 15 retrieves LI and CI from 
memory 16, and generates new values LO (new smoothed pixel value) and CO (new 
time constant) that are stored in memory 1 6 to replace LI and CI respectively. As shown 
in Fig. 2, temporal processing unit 15 transmits the CO and DP values for each pixel to 
spatial processing unit 1 7 through the delay unit 18. 

The capacity of memory 16 assuming that there are R pixels in a frame, 
and therefore 2R pixels per complete image, must be at least 2R(e+f) bits, where e is the 
number of bits required to store a single pixel value LI (preferably eight bits), and f is the 
number of bits required to store a single time constant CI (preferably 3 bits). If each 
video image is composed of a single frame (progressive image), it is sufficient to use 
R(e+f) bits rather than 2R(e+f) bits. 

Spatial processing unit 17 is used to identify an area in relative movement 
in the images from camera 13 and to determine the speed and oriented direction of the 
movement. Spatial processing unit 17, in conjunction with delay unit 18, cooperates 
with a control unit 19 that is controlled by clock 20, which generates clock pulse HP at 
the pixel frequency. Spatial processing unit 17 receives signals DPy &id CO„ (where i 
and j correspond to the x and y coordinates of the pixel) from temporal processing unit 
15 and processes these signals as discussed below. Whereas temporal processing unit 15 
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processes pixels within each frame, spatial processing unit 17 processes groupings, of 
pixels between the frames. 

Fig. 5 diagrammatical^ shows the temporal processing of successive 
corresponding frame sequences TO,. TR 2 , TR 3 and the spatial processing in these frames 
of a pixel PI with coordinates x, y, at times t„ t 2 , and t 3 . A plane in Fig. 5 corresponds to 
the spatial processing of a frame, whereas the superposition of frames corresponds to the 
temporal processing of successive frames. 

Signals DP ;j and CO i} from temporal processing unit 15 are distributed by 
spatial processing unit 17 into a first matrix 21 containing a number of rows and columns 
much smaller than the number of lines L of the frame and the number of pixels M per 
line. Matrix 21 preferably includes 21 + 1 lines along the y axis and 2mH columns along 
the x axis (in Cartesian coordinates), where / and m are small integer numbers. 
Advantageously, / and m are chosen to be powers of 2, where for example / is equal to 2* 
and m is equal to 2 b , a and b being integer numbers of about 2 to 5, for example. To 
simplify the drawing and the explanation, m will be taken to be equal to / (although it 
may be different) and m=/=2 3 =8. In this case, matrix 21 will have 2x8 + 1 = 17 rows 
and 17 columns. Fig. 4 shows a portion of the 17 rows Y„, Y,,... Y 13 , Y 16 , and 17 
columns X 0 , Xj, ... X,5, X J6 which form matrix 21. 

Spatial processing unit 17 distributes into / x m matrix 21 the incoming 
flows of Dp ijt and CO jt from temporal processing unit 15. It will be appreciated that only 
a subset of all DP ij( and CO ijt values will be included in matrix 21, since the frame is much 
larger, having L lines and M pixels per row (e.g., 312.5 lines and 250-800 pixels), 
depending upon the TV standard used. 

In order to distinguish the L x M matrix of the incoming video signal from 
the I x m matrix 21 of spatial processing unit 17, the indices i and j will be used to 
represent the coordinates of the former matrix and the indices x and y will be used to 
represent the coordinates of the latter. At a given instant, a pixel with an instantaneous 
value PI ijt is characterized at the input of the spatial processing , unit 17 by signaMP* 
and CO*. The (2/+1 ) x (2m + 1) matrix 21 is formed by scanning each of the LxM 
matrices for DP and CO. 
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In matrix 21, each pixel is defined by a row number between 0 and 16 
(inclusive), for rows Y„ to Y 16 respectively, and a column number between 0 and 16 
(inclusive), for columns X, to X, 6 respectively, in the case in which / « m = 8. In this 
case, matrix 21 will be a plane of 17 x 17 = 289 pixels. 

In Fig. 4, elongated horizontal rectangles Y 0 to Y| 6 (only four of which 
have been shown, i.e., Y„, Y,, Yis and Y w ) and vertical lines Xo to X, 6 (of which only 
four have been shown, i.e., Xo, X,, X, 5 and X, 6 ) illustrate matrix 21 with 17 x 17 image 
points or pixels having indices defined at the intersection of an ordinate row and an 
abscissa column. For example, the P 88 is at the intersection of column 8 and row 8 as 
illustrated in Fig. 4 at position e, which is the center of matrix 21. 

In response to the HP and BL signals from clock 20 (Fig. 2), a rate 
control or sequencing unit 1 9: i) generates a line sequence signal SL at a frequency equal 
to the quotient of 13.5 MHZ (for an image with a corresponding number of pixels) 
divided by the number of columns per frame (for example 400) to delay unit 18, ii) 
generates a frame signal SC, the frequency of which is equal to the quotient 13.5/400 
MHZ divided by the number of rows in the video image, for example 312.5, hi) and 
outputs the HP clock signal. Blanking signal BL is used to render sequencing unit 19 
non-operational during synchronization signals in the input image. 

A delay unit 18 carries out the distribution of portions of the L x M 
matrix into matrix 21. Delay unit 18 receives the DP, CO, and incoming pixel S(PI) 
signals, and distributes these into matrix 21 using clock signal HP and line sequence and 

column sequence signals SL and SC. 

In order to form matrix 21 from the incoming stream of DP and CO 
signals, the successive row, Y 0 to Y l6 for the DP and CO signals must be delayed as 
follows: 

row Yo - not delayed; 

row Yi - delayed by the duration of a frame line TP; 

row Y 2 - delayed by 2 TP; ;;• 

and so on until 

row Y i6 - delayed by 1 6 TP . 
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The successive delays of the duration of a frame row TP, are carried^ 
in a cascade of sixteen delay circuits r,,r 2 ,.., l6 that serve rows Y,.Y 2 ...Y 16 , respectively 
row Y. bemg served directly by the DP and CO signals without any delay upon arriving 
from temporal processing unit 15. All delay circuits r,,r 2( ...r 16 may be built up by a delay 
line with sixteen outputs, the delay imposed by any section thereof between two 
successive outputs being constant and equal to TP. 

Rate control unit 19 controls the scanning of the entire L x M frame 
matrix over matrix 21. The circular displacement of pixels in a row of the frame matrix 
on the 17x17 matrix, for example from Xo to X I6 on row Y„, is done by a cascade of 
sixteen shift registers d on each of the 17 rows from Y 0 to Y I6 (giving a total of 16 x 17 
- 272 shift registers) placed in each row between two successive pixel positions, namely 
the register d 0I between positions PI 00 and PI 0I register d 02 between positions PI 0I , and 
PIo 2 , etc. Each register imposes a delay TS equal to the time difference between two 
successive pixels in a row or line, using column sequence signal SC. Because rows /,, / 2 
... In in a frame TR, (Fig. 1), for S(PI) and for DP and CO, reach delay unit 18 shifted'by 
TP (complete duration of a row) one after the other, and delay unit 18 distributes them 
with gradually increasing delays of TP onto rows Y 0 , Y, ... Y l7 , these rows display the 
DP and CO signals at a given time for rows IJ2 ... /„ in the same frame portion. 
Similarly in a given row, e.g., /,, successive pixel signals a,,, a, 2 ... arrive shifted by TS 
and shift registers d impose a delay also equal to TS. As a result, the pixels of the DP 
and CO signals in a given row Y 0 to Y, 6 in matrix 21, are contemporary, i.e., they 
correspond to the same frame portion. 

The signals representing the COs and DPs in matrix 21 are available at a 
given instant on the 16 x 17 = 272 outputs of the shift registers, as well as upstream of 
the registers ahead of the 17 rows, i.e., registers da,, d,,.... d 16 .,, which makes a total of 
16x 17+ 17= 17x 17 outputs for the 17 x 17 positions Po.o,Po.«,...P g .g...P, 6 . I6 . 

In order to better understand the process of spatial processing, the system 
will be described %ith respect to a small matrix M3 containing 3 rows and 3 columns 
where the central element of the 9 elements thereof is pixel e with coordinates x = 8, y = 
8 as illustrated below: 
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d e f (M3) 
g h i 

5 In matrix M3, positions a, b, c, d, f, g, h, i around the central pixel e 

correspond to eight oriented directions relative to the central pixel. The eight directions 
may be identified using the Freeman code illustrated in Fig. 6, the directions being coded 
0 to 7 starting from the x axis, in steps of 45°. In the Freeman code, the eight possible 
oriented directions, may be represented by a 3-bit number since 2 3 = 8. 
10 Considering matrix M3, the 8 directions of the Freeman code are as 

follows: 

3 2 1 

4 e 0 

5 6 7 

15 

Returning to matrix 21 having 17 x 17 pixels, a calculation unit 17a 
examines at the same time various nested square second matrices centered on e, with 
dimensions 15x15, 13x13, 11 x 11, 9 x 9, 7 x 7, 5 x 5 and 3 x 3, within matrix 21, the 
3x3 matrix being the M3 matrix mentioned above. Spatial processing unit 17 

20 determines which matrix is the smallest in which pixels with DP = 1 are aligned along a 
straight line which determines the direction of movement of the aligned pixels. 

For the aligned pixels in the matrix, the system determines if CO varies on 
each side of the central position in the direction of alignment, from +a in an oriented 
direction and -a in the opposite oriented direction, where l<a<N. For example, if 

25 positions g, e, and c of M3 have values -1, 0, +1, then a displacement exists in this matrix 
from right to left in the (oriented) direction 1 in the Freeman code (Fig. 6). However, 
positions g, e, and c must at the same time have DP = 1 . The displacement speed of the 
pixels in motion is grt§;er when the matrix, among the 3 x 3 to 15 x 15 nested matrices, 
in which CO varies from +1 or -1 between two adjacent positions along a direction is 

30 larger. For example, if positions g, e, and c in the 9 x 9 matrix denoted M9 have values - 
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1, 0, +1 in oriented direction 1, the displacement will be fester than for values -1, 0, +1 in 
3 k 3 matrix M3 (Fig. 7). The smallest matrix for which a line meets the test of DP=1 for 
the pixels in the line and CO varies on each side of the central position in the direction of 
alignment, from +a in an oriented direction and -a in the opposite oriented direction, is 
5 chosen as the principal line of interest. 

Within a given matrix, a greater value of ±CO indicates slower 
movement. For example, in the smallest matrix, i.e., the 3x3 matrix, CO=±2 with DPs=l 
determines subpixel movement i.e. one half pixel per image/and CO=±3, indicates 
slower movement, i.e. one third of a pixel per image. In order to reduce the calculation 
10 power in the system and to simplify the hardware, preferably only those values of CO 
which are symmetrical relative to the central pixel are considered. 

Since C0 is presented as a power of 2 in a preferred embodiment, an 
extended range of speeds may be identified using only a few bits for CO, while still 
enabling identification of relatively low speeds. Varying speed may be detected because, 
15 for example -2, 0, +2 in positions g, e, c in 3 x 3 matrix M3 indicates a speed half as fast 
as the speed corresponding to 1, 0, +1 for the same positions in matrix M3. 

Two tests are preferably performed on the results to remove uncertainties. 
The first test chooses the strongest variation, in other words the highest time constant, if 
there are variations of CO along several directions in one of the nested matrices. The 
20 second test arbitrarily chooses one of two (or more) directions along which the variation 
of CO is identical, for example by choosing the smallest value of the Freeman code, in 
the instance when identical lines of motion are directed in a single matrix in different 
directions. This usually arises when the actual direction of displacement is approximately 
between two successive coded directions in the Freeman code, for example between 
25 directions 1 and 2 corresponding to an (oriented) direction that can be denoted 1.5 (Fig. 
6) of about 67.5' with the x axis direction (direction 0 in the Freeman code). 

The scanning of an entire frame of the digital video signal S preferably 
occurs in the following sequence. The %st group of pixels considered is the first 17 
rows or lines of the frame, and the first 17 columns of the frame. Subsequently, still for 
30 the first 17 rows of the frame, the matrix is moved column by column from the left of the 
frame to the right, as shown in Fig. 5, i.e., from portion TM, at the extreme left, then 
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TM 2 offset by one column with respect to TM„ until TM M (where M is the number of 
pixels per frame line or row) at the extreme right. Once the first 17 rows have been 
considered for each column from left to right, the process is repeated for rows 2 to 18 in 
the frame This process continues, shifting down one row at a time until the last group 
of lines at the bottom of the frame, i.e.. lines L - 16 ... L (where L is the number of lines 

per frame) are considered. 

Spatial processing unit 17 generates the following output signals for each 
pixel- i) a signal V representing the displacement speed for the pixel, based upon the 
amplitude of the maximum variation of CO surrounding the pixel, the value of which may 
, be for example, represented by an integer in the range 0 - 7 if the speed is in the form of 
a power of 2. and therefore may be stored in 3 bits, ii) a signal DI representmg the 
direction of displacement of the pixel, which is calculated from the direction of maxrmum 
variation, the value of DI being also preferably represented by an integer in the range 0 - 
7 corresponding to the Freeman code, stored in 3 bits, iii) a binary validation signal VL. 
5 which indicates whether the result of the speed and oriented direction is valid, m order to 
be able to distinguish a valid output with V - 0 and DI = 0, from the lack of an output 
due to an incident, this signal being 1 for a valid output or 0 for an invalid output, >v) a 
time constant signal CO, stored in 3 bits, for example, and v) a delayed video stgnal SR 
consisting of the input video signal S delayed in the delay unit 18 by 16 consecutive l,n« 
0 durations TR and therefore by the duration of the distribution of the signal S «. the 17x 
17 matrix 21, in order to obtain a video signal timed to matrix 21, which may be 
displayed on a television set or monitor. Also output are the clock signal HP, hne 
sequence signal SL and column sequence signal SC from control unit 19. 

Nested hexagonal matrices (Fig 8) or an inverted L-shaped matnx (F,g. 9) 
25 may be substituted for the nested rectangular matrices in Fig, 4 and 7. In the case shown 
in Fig 8 the nested matrices (in which only the most central matrices MRI and MR2 
have been shown) are all centered on point MRO which corresponds to the central potnt 
* of matrices M3, M9 in Fig. 7. The advantage of a hexagonal matrix system .s tha* U 
" allows the use of oblique coordinate axes x„ y., and a breakdown into triangles w,th 
30 identical sides, to carry out an isotropic speed calculation. 
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,C ^ , „■■ . ^ " R& 9 " C ° mP0Sed ° f " ** r ° W W »d a sing.. column 
<C) «*, from the ce„,ral position MR. i„ which the ^ ^ Dp 

respectively are equal ,o "P for DP and increase or decrease by one unit for CO if 
movement occurs. ' 
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identical in all positions (boxes) in column C., and the binary signal DP is eo.ua, ,o fL an 
positions ,„ row L^om .he originMP,. wilh to value c0>> ^ ,„ 
CO is equa, ,o CO. + , or ., inCusive. u movement is ^ directibn of ^ y 
coor ,„a,e, the CO signa, is identic*, in a,, prions (boxes) in row L., and ^ 
agna, DP ,s equal ,o , in a„ positions in co,umn C_ from the origin MR, the ^ 
CO, up to the position in which CO is equa, «p CO.. +1 or ., M , It movement is 
obhque re,a„ve to the x and y coordinates, the binary signa, DP is equa, to , and CO is 
equal ,„ CO. I. positions (boxes, of L. and in positions (boxes) of C., the s,ope being 
determined by the perpendicuUr to the ,i„e passing through the two positions in which 
the signal CO. changes by the value of one unit, the DP signal always being equa, to 1 

Fig. 9 shows the case in which DP = 1 and CO. changes value by one unit 
« <he two specific positions U, and C and indicates the corresponding slope P P to aO 
cases, the displacement speed is a taction of the position in which CO changes value by 
one uni, ,f CO changes by one unit in K or C. only, i, corresponds to the value of the 
CO variation position. ,f CO changes by one uni, in a position in L. and in a position in 
C the speed is proportional to the distance between MR. and (intersection of the line 
perpendicular to C.-L. passing through MR.). 

Fig. 10 shows an imaging device with sensors located at the intersections 
of concentric lines c and radial lines d that correspond to the rows and columns of a 
rectangular matrix imaging device. The operation of such an imaging device is 
controlled by a circular scanning sequencer. ,„ this embodiment, angular sector shaped n 
x n matrices MC are formed, (a 3x3 matrix MC3 and a 5x5 matrix MC5 are shown) and 
except for sequencing differences, the matrices are processed fentical , 0 the square 
matrix embodiments discussed above. 
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As shown in Figs. 1 1-16, spatial and temporal processing unit 11 is used 
in connection with a histogram processor 22a for identifying objects within the input 
signal based upon user specified criteria for identifying such objects. A bus Z-Z, (See 
Figs. 2, 11 and 12) transfers the output signals of spatial and temporal processing unit 11 
5 to histogram processor 22a. Histogram processor 22a generates composite output signal 
ZH which contains information on the areas in relative movement in the scene. 

Referring to Fig. 12, histogram processor 22a includes a bus 23 for 
communicating signals between the various components thereof, for receiving input 
commands from a controller 42 and for transmitting output signals to controller 42. 
10 Histogram formation and processing blocks 24 - 29 receive the various input signals, i.e., 
delayed digital video signal SR. speed V, oriented directions (in Freeman code) DI, time 
constant CO, first axis x(m) and second axis y(m), which are discussed in detail below. 
The function of each histogram formation block is to enable a histogram to be formed for 
the domain associated with that block. For example, histogram formation block 24 
1 5 receives the delayed digital video signal SR and enables a histogram to be formed for the 
luminance values of the video signal. Since the luminance of the signal will generally be 
represented by a number in the range of 0-255, histogram formation block 24 is 
preferably a memory addressable with 8 bits, with each memory location having a 
sufficient number of bits to correspond to the number of pixels in a frame. 
20 Histogram formation block 25 receives speed signal V and enables a 

histogram to be formed for the various speeds present in a frame. In a preferred 
embodiment, the speed is an integer in the range 0-7. Histogram formation block 25 is 
then preferably a memory addressable with 3 bits, with each memory location having a 
sufficient number of bits to correspond to the number of pixels in a frame. 

Histogram formation block 26 receives oriented direction signal DI and 
enables a histogram to be formed for the oriented directions present in a frame. In a 
preferred embodiment, the oriented direction is an integer in the range 0-7, 
corresponding to the Freeman code. Histogram formation t^ck 26 is then preferably a 
memory addressable with 3 bits, with each memory location having a sufficient number 
30 of bits to correspond to the number of pixels in a frame. 
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HOW*, formation block 27 receives time constim, signal CO and 
enables a histogram ,o be fonned for the toe consents of the pixels in a frame In a 
preferred embodiment, the time constant is an integer in the range 0-7. Histogram 
formation block 27 is then preferably a memory addressable with 3 bits with each 
memory iocation having a sufficient number of bits to correspond to the number of pixels 
in a frame. . 

Histogram formation blocks 28 and 29 receive the x and y positions 
respectively of pixels for which a histogram is t0 be fonned, and form histograms fo, 
such p,xe.s, as discussed in greater detail below. Histogram formation Mock 28 is 
preferably addressable with the number of bits corresponding to the number of pixels in a 
line, w„h each memory location having a sufficient number of bits to correspond to the 
number of .iocs in a frame, and histogram formation block 29 is preferably addressable 
wth the number of bits corresponding to the number of lines in a frame, with each 
memory location having a sufficient number of bits to correspond to the number of pixels 
5 ma line. 

Referring to Figs. 12 and 14, each of the histogram formation blocks 24 - 
29 has an associated validation block 30- 35 respectively, which generates a validation 
signal VI - V6 respectively. In general, each of the histogram formation blocks 24-29 is 
identical to the others and functions in the same manner. For simplicity, the invention 
wdl be described with respect to the operation of histogram formation block 25 i, being 
apprecated that the remaining histogram formation blocks operate in a like manner 
Htstogram formation block 25 inciudes a histogram forming p„ n i 0 „ 25a. which forms 
■he tastogram for that block, and a classifier 25b, for selecting the criteria of pixels for 
which the histogram is to be formed. Histogram forming portion 25a and classifier 25b 
operate under the control of computer software in an integrated circuit (not shown) to 
extract cettain limits of the histograms generated by the histogram formation block, and 
to control operation of the various components of the histogram formation units. 

Referring to Fig. 14, histogram forming portion 25a includes a memory „ 
100, wmch is preferably a conventional digital memory. In the case of histogram 
formation block 25 which forms a histogram of speed, memory ,00 is sued to have 
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addresses 0-7, each of which may store up to the number of pixels in an image. Between 
frames, memory 100 is initiated, i.e., cleared of all memory, by setting IniH in 
multiplexors 102 and 104. This has the effect, with respect to multiplexor 102 of 
selecting the "0" input, which is output to the Data In line of memory 100. At the same 
time, setting init=\ causes multiplexor 104 to select the Counter input, which is output to 
the Address line of memory 100. The Counter input is connected to a counter (not 
shown) that counts through all of the addresses for memory 100, in this case 
0<address<7. This has the effect of placing a zero in all memory addresses of memory 
100. Memory 100 is preferably cleared during the blanking interval between each frame. 
After memory 100 is cleared, the inii line is set to zero, which in the case of multiplexor 
102 results in the content of the Data line being sent to memory 100, and in the case of 
multiplexor 104 results in the data from spatial processing unit 117, i.e., the V data, 
being sent to the Address line of memory 100. 

Classifier 25b enables only data having selected classification criteria to be 
5 considered further, meaning to possibly be included in the histograms formed by 
histogram formation blocks 24-29. For example, with respect to speed, which is 
preferably a value in the range of 0-7, classifier 25b may be set to consider only data 
within a particular speed category or categories, e.g., speed 1, speeds 3 or 5, speed 3-6, 
etc Classifier 25b includes a register 106 that enables the classification criteria to be set 
20 by the user, or by a separate computer program. By way of example, register 106 will 
include, in the case of speed, eight registers numbered 0-7. By setting a register to »1», 
e g register number 2, only data that meets the criteria of the selected class, e.g., speed 
2 will result in a classification output of "1". Expressed mathematically, for any given 
register in which R(k) = b, where k is the register number and b is the boolean value 
25 stored in the register: 

Output= R(data(V)) 

So for a data point V of magnitude 2, the output of classifier 25b will be "1" only if 
R(2)=l . The classifier associated *ith histogram formation block 24 preferably has 256 
registers, one register for each possible luminance value of the image. The classifier 
30 associated with histogram formation block 26 preferably has 8 registers, one register for 
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each possible direction value. The classifier associated with histogram formation block 
27 preferably has 8 registers, one register for each possible value of CO. The classifier 
associated with histogram formation block 28 preferably has the same number of 
registers as the number of pixels per line. Finally, the classifier associated with histogram 
formation block 29 preferably has the same number of registers as the number of lines 
per frame. The output of each classifier is communicated to each of the validation blocks 
30-35 via bus 23, in the case of histogram formation blocks 28 an 29, through 
combination unit 36, which will be discussed further below. 

Validation units 30-35 receive the classification information in parallel 
from all classification units in histogram formation blocks 24 - 29. Each validation unit 
generates a validation signal which is communicated to its associated histogram 
formation block 24 - 29. The validation signal determines, for each incoming pixel, 
whether the histogram formation block will utilize that pixel in forming it histogram. ' 
Referring again to Fig. 14, which shows histogram formation block 25, validation unit 31 
includes a register block 108 having a register associated with each histogram formation 
block, or more generally, a register associated with each data domain that the system is 
capable of processing, in this case, luminance, speed, direction, CO, and x and y position. 
The content of each register in register block 108 is a binary value that may be set by a 
user or by a computer controller. Each validation unit receive via bus 23 the output of 
each of the classifiers, in this case numbered 0 ... p, keeping in mind that for any data 
domain, e.g., speed, the output of the classifier for that data domain will only be "1" if 
the particular data point being considered is in the class of the registers set to "1" in the 
classifier for that data domain. The validation signal from each validation unit will only 
be "1" if for each register in the validation unit that is set to "1", an input of "1" is 
received from the classifier for the domain of that register. This may be expressed as 
follows: 



oul = .('*> + Rego). (//>, + Reg,) ... (7/i, + Reg n )(/w 0 + m/+...w fl ) 
where Rego is the register in the validation unit associated with input in* Thus, using the 
30 classifiers in combination with validation units 30 - 35, the system may select for 
processing only data points in any selected classes within any selected domains. For 
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example, the system may be used to detect only data points having speed 2, direction 4, 
and luminance 125 by setting each of the following registers to "1": the registers in the 
validation units for speed, direction, and luminance, register 2 in the speed classifier, 
register 4 in the direction classifier, and register 125 in the luminance classifier. In order 
to form those pixels into a block, the registers in the validation units for the x and y 
directions would be set to " 1 " as well. 

Referring again to Fig. 13, validation signal V2 is updated on a pixel-by- 
pixel basis. If, for a particular pixel, validation signal V2 is "1", adder 110 increments 
the output of memory 100 by one. If, for a particular pixel, validation signal V2 is "0", 
adder 100 does not increments the output of memory. In any case, the output of adder 
100 is stored in memory 100 at the address corresponding to the pixel being considered. 
For example, assuming that memory 100 is used to form a histogram of speed, which 
may be categorized as speeds 0-7, and where memory 100 will include 0-7 
corresponding memory locations, if a pixel with speed 6 is received, the address input to 
multiplexor 104 through the data line will be 6. Assuming that validation signal V2 is 
"1", the content in memory at location 6 will be incremented. Over the course of an 
image, memory 100 will contain a histogram of the pixels for the image in the category 
associated with the memory. If, for a particular pixel, validation signal V2 is "0" because 
that pixel is not in a category for which pixels are to be counted (e g., because that pixel 
does not have the correct direction, speed, or luminance), that pixel will not be used in 
forming the histogram. 

For the histogram formed in memory 100, key characteristics for that 
histogram are simultaneously computed in a unit 112. Referring to Fig. 14, unit 112 
includes memories for each of the key characteristics, which include the minimum (MIN) 
of the histogram, the maximum (MAX) of the histogram, the number of points (NBPTS) 
in the histogram, the position (POSRMAX) of the maximum of the histogram, and the 
number of points (RMAX) at the maximum of the histogram. These characteristics are 
determined in parallel with the formation of the histogram as follows: 

For each pixel with a validation signal V2 of f, P: 
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(a) if the data value of the pixel < MIN (which is initially set to. the 
maximum possible value of the histogram), then write data value in MIN; 

(b) if the data value of the pixel > MAX (which is initially set to the 
minimum possible value of the histogram), then write data value in MAX; 

(c) if the content of memory 1 00 at the address of the data value of 
the pixel > RMAX (which is initially set to the minimum possible value of the histogram), 
then i) write data value in POSRMAX and ii) write the memory output in RMAX, 

(d) increment NBPTS (which is initially set to zero) 

At the completion of the formation of the histogram in memory 100 at the 
end of each frame, unit 112 will contain important data characterizing the histogram. 
The histogram in each memory 100, and the characteristics of the histogram in units 1 12 
are read during the scanning spot of each frame by controller 42, and the memories 100 
are cleared and units 1 12 are re-initialized for processing the next frame. 

The system of the invention includes a semi-graphic masking function to 
select pixels to be considered by the system. Fig. 17 shows a typical image 53 consisting 
of pixels arranged in a Q x R matrix, which is divided into sub-matrices 5 1 each having a 
dimension of s x /, wherein each s x / sub-matrix includes s.x t number of pixels of the 
image. Each sub-matrix shown in Fig. 17 is a 3x4 matrix. In a preferred embodiment, 
*=9 and r=12, although any appropriate sub-matrix size may be used, if desired, including 
lxl. Referring to Fig. 12, histogram processor 22a includes a semi-graphic memory 50, 
which includes a one-bit memory location corresponding to each s x / matrix. For any 
given sub-matrix 51, the corresponding bit in memory 50 may be set to "0", which has 
the effect of ignoring all pixels in such sub-matrix 50, or may be set to " 1 " in which case 
all pixels in such sub-matrix will be considered in forming histograms. Thus, by using 
semi-graphic memory 50, it is possible to limit those areas of the image to be considered 
during histogram formation. For example, when an image of a road taken by a camera 
facing forward on a vehicle is used to detect the lanes of the road, the pixel information 
of the road at the farthest distances from the c^iera generally does not contain useful 
information. Accordingly, in such an application, the semi-graphic memory is used to 
mask off the distant portions of the road by setting semi-graphic memory 50 to ignore 
such pixels. Alternatively, the portion of the road to be ignored may be masked by 
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setting the system to track pixels only within a detection box that excludes the undesired 
area of the screen, as discussed below. 

In operation, for any pixel under consideration, an AND operation is run 
on the validation signal for such pixel and the content of semi-graphic memory 50 for the 
sub-matrix in which that pixel is located. If the content of semi-graphic memory 50 for 
the sub-matrix in which that pixel is located contains "O", the AND operation will yield a 
"0" and the pixel will be ignored, otherwise the pixel will be considered in the usual 
manner. It is foreseen that the AND operation may be run on other than the validation 
signal, with the same resultant functionality. Also, it is foreseen that memory 50 may be 
a frame size memory, with each pixel being independently selectable in the semi-graphic 
memory. This would enable any desired pixels of the image to be considered or ignored 
as desired. Semi-graphic memory 50 is set by controller 42 via data bus 23. 

Fig. 16 shows an example of the successive classes Ci, C 2 ...C n .i, C„, each 
representing a particular velocity, for a hypothetical velocity histogram, with their being 
categorization for up to 16 velocities (15 are shown) in this example. Also shown is 
envelope 38, which is a smoothed representation of the histogram. 

In order to locate the position of an object having user specified criteria 
within the image, histogram blocks 28 and 29 are used to generate histograms for the x 
and y positions of pixels with the selected criteria. These are shown in Fig. 13 as 
histograms along the x and y coordinates. These x and y data are output to moving area 
formation block 36 which combines the abscissa and ordinate information x(m) 2 and 
y(m) 2 respectively into a composite signal xy(m) that is output onto bus 23. A sample 
composite histogram 40 is shown in Fig. 13. The various histograms and composite 
signal xy(m) that are output to bus 23 are used to determine if there is a moving area in 
the image, to localize this area, and/or to determine its speed and oriented direction. 
Because the area in relative movement may be in an observation plane along directions x 
and y which are not necessarily orthogonal, as discussed below with respect to Fig. 18, a 
data change block 37 may be used to convert the x and y data to orthogonal c$:>rdinates. 
Data change block 37 receives orientation signals x(m)i and y(m)i for x(m) 0 and y(m) 0 
axes, as well as pixel clock signals HP, line sequence and column sequence signals SL 
and SC (these three signals being grouped together in bundle F in Figs. 2, 4, and 10) and 
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generates the orthogonal x( m ), and y( m ,, signa)s that „ . 
blocks 28 and 29 respectively. ^ f °™ a " on 

In order to process pixels only within a user-defined area, the x-direcri™, 

28 may be prograramed to 

5 prxels defined by boundaries, i.e. XMTN and XMAX n,i. • . 

XMTN and XMAX ™. ■ "comphshed by setting the 

XMTN and XMAX values „ a user-programmable memory in x-direction histogram 

«* no, be processed. Similarly, y-direcion bhK>srm ^ ^ , ^ 

process pucels o„,y ta a Cass of pixels defined by boundaries YMIN and YMAX « • 
by setting the YMIN and YMAX values in a user-prograr^ , 
y-chrecon rnstogram formation unit 29 or in „„ear combination units 30-35 ThL 
^.em can process pixeis on,y in a defined rec,a„g,e by setting the XMm and 
^AX, and YMIN an, VMAX values as desired. Of course, the classification cri, ri 

T m T' m ; ria frcra ,he o,her ws,osram ^ * - - 

*™m fcsograms of on,y se.ec.ed classes of pixels in seieaed domains within the se.ec.ed 
bits tore ^ XMAX memory locations have a sufficient number of 

«s . represent the maximum number of pixe,s in the x dimension of the nnage „„ der 
s,era,.o„, and me YM.N and VMAX memo,y Nations have a sufficient number 

«s to represent the maximum number of pixe.s in the y dimension the image under 
™ de rat ,o. As discussed fcrther be,ow, the x and y axes may be rotated m order ,o 

XMm, XM AX, YMIN and YMAX memory locations have a sufficient number of bits to 
represent the maximum number of pixels along .he diagonal of the image under 
cons,de rat , (lne toce from „ 0rigin „ (o „ stop „ ^ pjg ^ £ r 

rnay be used to search within a user-defined rectangle ,ong a user-defined ro J axis 

Inorderforapixe.PI(a,b) to be considered in the formation of x and y 
dtrec ,on h.,og™, whether on the orthogona, coordinate axes or along «a,ed axes, 

.nese .ests may be ANDed with the validafion signa, so tha. if the conditions are no. 
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satisfied, a logical "0" is ANDed with the validation signal for the pixel under 
consideration, thereby avoiding consideration of the pixel in the formation of x and y 
direction histograms. 

Fig. 13 diagrammaticaily represents the envelopes of histograms 38 and 
39, respectively in x and y coordinates, for velocity data. In this example, x M and y M 
represent the x and y coordinates of the maxima of the two histograms 38 and 39, 
whereas U and l b for the x axis and U and l d for the y axis represent the limits of the range 
of significant or interesting speeds, 1, and U being the longer limits and l b and U being the 
upper limited of the significant portions of the histograms. Limits 1„ l b , U and l d may be 
set by the user or by an application program using the system, may be set as a ratio of the 
maximum of the histogram, e.g., xm/2, or may be set as otherwise desired for the 
particular application. 

The vertical lines L, and L b of abscissas 1, and l b and the horizontal lines 
L c and L d of ordinals l c and l d form a rectangle that surrounds the cross hatched area 40 
of significant speeds (for all x and y directions). A few smaller areas 41 with longer 
speeds, exist close to the main area 40, and are typically ignored. In this example, all 
that is necessary to characterize the area with the largest variation of the parameter for 
the histogram, the speed V in this particular case, is to identify the coordinates of the 
limits l a , l b , l c and l d and the maxima X M and Y M , which may be readily derived for each 
histogram from memory 1 00, the data in units 1 12, and the xy(m) data block. 

JThus, the system of the invention generates in real time, histograms of 
jach of the parameters being detected. Assuming that it were desired to identify an 
object with a speed of "2" and a direction of "4", the validation units for speed and 
direction would be set to "1", and the classifiers for speed "2" and direction "4" would be 
set to "1". In addition, since it is desired to locate the object(s) with this speed and 
direction on the video image, the validation signals for histogram formation blocks 28 
and 29, which correspond to the x and y coordinates, would be set to "1" as well. In this 
way, histogram formation blocks 28 and 29 wouhfform histograms of only the pixels 
with the selected speed and direction, in real-time. Using the information in the 
histogram, and especially POSRMAX, the object with the greatest number of pixels at 
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*e *<ected speed a„ d direction „„,„ be 

More M . fc histogram f orma , ioa blocks ran loca|ize ^ 'J*** 

use,se.e«ed criteria, a„ d may prooW „ ^^^1"^ 
A*-** , he infonm rio„ ^ „e ^ J £l 

near or rem0 , e from spatial ^ ^ ^ j _ % >■ wh,ch may 

and vertical MN of *i, • y ned h Y the horizontal 

rotation of the analysis axis to d«*r' • , W 3 ° f Usin S 

axis system for considerat on bv the x w v i,;** * 

n . _ y the X and y hl *ogram formation units 28 and 29 The 

rotated axes), forming histograms on the rotated axes »„h . " u- 
i direction, etc. r0tated ^ searching usmg velocity, 

follow , ^ diSCUSS6d ^ hi5t0gram fo ^ion unit calculates the 
following values for its respective histogram. 

r . ' ~ MIN, MAX, NBPTS, RMAX, POSRMAX 

syir i at the : v : ,ues are ca,cu,ated in * « — the 

astern to rap.dly .dentily ,i nes on an image. While this may he accomplished in a 
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number of different ways, one of the easier methods is to calculate R, where R 
=NBPTS/RMAX, i.e., the ratio of the number of points in the histogram to the number 
of points in the maximal line. The smaller this ratio, i.e., the closer R approaches 1, the 
more perpendicularly aligned the data points under consideration are with the scanning 



axis. 



Fig. 15A shows a histogram of certain points under consideration, where 
the histogram is taken along the x-axis, i.e., projected down onto the x-axis. In this 
example, the ratio R, while not calculated, is high, and contains little information about 
the orientation of the points under consideration. As the x-axis is rotated, the ratio R 
increases, until, as shown in Fig. 15B, at approximately 45" the ratio R would reach a 
maximum. This indicates that the points under consideration are most closely aligned 
perpendicular to the 45° x-axis. In operation, on successive frames, or on the same 
frame if multiple x-direction histogram formation units are available; it is advantageous 
to calculate R at different angles, e.g., 33.75° and 57.25' (assuming the axes are limited 
15 to 16 degrees of rotation), in order to constantly ensure that R is at a minimum. For 
applications in which it is desirable to detect lines, and assuming the availability of 16 x- 
direction histogram formation units, it is advantageous to carry out the calculation of R 
simultaneously along all possible axes to determine the angle with the minimum R to 
determine the direction of orientation of the line. Because the x and y axes may be 
rotated independently, the x and y- histogram formation units ' are capable of 
• simultaneously independently detecting lines, such as each side line of a road, in the same 
manner. 

As discussed above, the system of the invention may be used to search for 
objects within a bounded area denned by XMIN, XMAX, YMIN and YMAX. Because 

25 moving object may leave the bounded area the system preferably includes an anticipation 
function which enables XMIN, XMAX, YMIN and YMAX to be automatically 
modified by the system to compensate for the speed and direction of the target. This is 
accomplished by determining values for O-MVT, corresponding to orientation 
(direction) of movement of the target within the bounded area using the direction 

30 histogram, and I-MVT, corresponding to the intensity (velocity) of movement. Using 
these parameters, controller 42 may modify the values of XMIN, XMAX, YMIN and 
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J t, e.g., a Ime, to, is bemg tracked based upon its axis of rotation, will be chanri 

and to enable user input of commands and controls such as wit a ■ * 

vanous parameters ,b at may be se, in the system and , ^1 ~ T^' 

» — units 3,,,, tbe coL, of ^^^^t^ 

%«« . 12, ,n order to analyze the results of the histogram formation process In 
M a^m ge„ eral controIler 42 My _ ^ ^ ^ ^ ^ PC- , 

Figs. 1 8-?? shows an example of use of the system of the invention to 
erform certam .unctions use*, for automatic vehicle cruise-control, inc^ Z 
of ,he hnes on a road, detection of ,a„e S on a road in which the vehicie is L 7Z 
^„o„ of verges passing or being passed by the vehide under JT 

^ S ' de ° f ^ Md ' »"< 2M «■» Hgh. side of the lane. h , 

2 2 Z: ,h ;^.° f 7 d ^— . mthecaseofmergingJesand' 
», lanes the hne bound.ng the side of the road may end temporary, with one or more 
3 Imesmergmg w,th or diverging from the lane. 
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The invention will be described with respect to detection of a solid white 
line 204 on the right side of the lane. Referring to Fig. 19, controller 42 is initially placed 
m an acquisition mode (206). Since a white line is defined by high luminance, controller 
42 sets the registers 106 in the luminance linear combination Unit to detect high 
luminance levels (208). Other features that may be associated with the right side line are 
motion (DP) and direction (DI). It is foreseen that these and other features associated 
with the line may be tracked, if desired. The actual luminance levels will vary depending 
upon various factors, such as ambient lighting, time of day, weather conditions, etc 
Keeping in mind that controller 42 is able to access the histogram calculated for 
luminance from histogram formation unit 24, controller 42 may use a threshold or other 
desired technique to select the desired luminances to search for the white line, e.g., 
selecting the top 15% of luminance values for consideration. 

This example will assume that only a single x-dimension histogram 
formation unit is available for use. As such, the detection of the angle of a line is 
accomplished by sweeping, through a number of angles until the value of R, or any other 
desired parameter indicating the alignment of the line is achieved. We multiple x- 
dimension histogram formation units available, the calculation of R at different angles 
could be accomplished simultaneously. 

The right side line is normally located within some fixed area to the right 
of the vehicle angling inward on the video image. Accordingly, controller 42 sets the x- 
axis to a starting angle normally associated with the right side line (210) (the Initial 
Search Axis shown in Fig. 18), and sets the positions of XMIN, XMAX, YMIN and 
YMAX to cover a rectangular area to the right of the vehicle normally associated with 
the right line (212) (initial values of XMIN, XMAX, YMIN and YMAX are shown in 
Fig. 18). In the search mode, during which time the line is being acquired, XMIN 
XMAX, YMIN and YMAX are set to cover a wider area than normal in order to 
maximize the likelihood of detection of the line. 

In order to confirm detection of the line, the controller runs one or more 
tests on the histogram at the desired luminance levels in'the desired area. For example, 
the line would normally be expected to have a certain length. Accordingly, the system 
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ofc for a threshoid „,„„,„„, number of pb(e|s u ^ 

« ,s constdered as being acquired. ff te ^ „ „„, ^ ^ ^ 

up .0 two P os,t, 0 „, 22,' in the prefer embody,, , 0 the « and right Z 
aarung x-axis una the target is acquired. 

■2,7 V , - °" Ce 1,38 aCq " ired ' *• ~ a tract mode 

0.7,. ,„ the trade mode, ^ XMAX, YMTN and YMAX are set to coveTa 
~ area than during the acquisition mo de to more Cose, cover the detected L 
). Du n „g the ^ mode the ^ „ normaUy oyer a ^ ^ - 

R ), wh,ch ensures tbatthe correct ang,e of the iine is constandy tracted. 
o the „e ,s iost, e.g„ because the controiier fails to detect a threshoid minimum nu J 
f P,xe.s a, RMAX M the search mode is rented. Once a tract is obtained 

rr er42may cons,an,,y moni,or ,he j ~ h — ; - - *— ~t 

update the Iummance levels associated with the line (224). 

searchab, ■ "T^ * ** *" "*" d "*« "» <*» «*■* '»« are 

detect hue and saturate when used in connection with a coior earner* Were the 
ystem acqu,r,ng a yeiiow iine, controiier 42 could se, the linear combination unit of the 
hue histogram formation unit to detect those hues associated with the color yeUow 
Were the system detecting a doubie line, there are a number of meUtods by which the 
Jtem may be augmented. A double line is charactered by a histogram having a peak 
— w„h each .inc. where, the peats are separated by a know, distance With 
•wo x-d.rec.ion histogram formation units, each could be set to tract a narrow 
rectanguiar area associated with a sing,, hne, wherein the narrow areas are separated 
from each other by the Known distance. Both histogram formation urits sweep 
amuitaneousiy, and only when both histogram fornuto units have detected a lin 
-uitaneousiy, has the doubie line been acquired. Actively, a single histogram 
format™ u„„ may be used to detect both lines in a simiiar manner over a period of two 
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frames, with a first line being detected during the first image or frame, and the second 
line detected during the next image or frame. This process is repeated to maintain track 
on the lines. During acquisition, the x-axis is preferably swept until a first line is acquired 
before attempting to acquire the second line. Since the second line may be positioned on 
either side of the first line, the search for the second line may be attempted at the known 
distance on each side of the first line until the second line is acquired. 

Alternatively, a single histogram formation unit may be used to detect the 
double line. In this embodiment the initial scanning area is set to be large enough to 
cover both lines. Since the histogram of the two lines will have two peaks, rather that 
using RMAX to detect the lines controller 42 may directly access and analyze the 
histogram of the selected area from memory 100 during the blanking interval, to locate 
the two peaks in the histogram that are associated with the two lines. As in the prior 
embodiment, each peak must have at least a minimum number of pixels or other 
characteristic indicative of a line to confirm acquisition of the double lines. Upon 
detecting the two peaks in the histogram, i.e., upon acquiring the lines, the system 
sweeps (rotates) the x-axis to maintain a maximum in the peaks of the histogram, i.e., to 

maintain track the lines. 

Detection of broken lines is similar and may be accomplished through a 
number of techniques. Broken lines would normally be associated with a time-varying 
histogram in which the number of points in the histogram and the peak of the histogram 
varies in a known cyclical manner. Thus, broken lines may normally be detected by any 
technique in which the histogram in the selected area associated with the line is time 
averaged, such as by having controller 42 perform any desired time averaging function 
on the histogram of the broken line, or by analyzing the peaks of the histogram of the 
line over time to detect the periodic fluctuations in the number of points in the histogram 
that would be associated with a broken line. 

Referring to Fig. 20, a single detection area 226 may be used to cover an 
area large enough to cover a broken line 228 and the gap between bfeiken lines, or some 
known multiple of lines. In this embodiment, detection of a broken line is similar to 
detection of a solid line, i.e., formation of a histogram of the pixels within the selected 
area and rotation of the axis of the detection area to maintain the line parallel to the 



25 



30 



WO 00/11609 

PCT/EP99/00425 

41 

detection axis, provided that the histogram of the *~ 

having fewerpixels than fcrasolid ^ ^ ™ "» 

Alternatively, refening to Figs 2IAanrt->iB <: 
may be used to detect a firs, portion of ,„ f I a ° a>0 " "» 230 

<■«*. the fact that each is sized to covert *" ^ ""■ JM 

Anguish a broken iine from a solid 1 T "* C ° n ' r< * ,er 42 * * * 

0 *■ and second ^T^^'^T ^ " "* * «" 
^ . • y mon,t °nng the cross-over of a i;„a «- 

detection area 230 to detection area ?i9 a u . e from 

-AX for each „ f detecti „Z 2 a^ * 2IB * wn " cn 3ho ws a graph o f 

and each is an .verse 
histograms over time, con.ro.ier 42 can detect ^ ^ ^ ° f 
■ * - * Pe«r , the pro^ r 1 ^ ^ 

threshold value t0 indicate de.ea.on of ,he line a , „■ t ' * 

™° main, n a m Mmum i„ Z^t^^T 
-ow the threshold value, the system ma, enter in, searl^e^" 
- qu ,re .he line. Wi,hi„ d=.ec.io„ area 212, controiler 42 m ^ZZ 
'oca.,on of the ,i„ e a„ d an oriema , ion ' I"*™* a 

detection area 232 t„ H« ^ crosses fr °m 

formation I * Ti " "* *" ^ ** ^™ 

As discussed above the x anH v t_ 
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As shown in Figs. 22 and 23, the system of the invention may be used to 
detect a vehicle passing or being passed by the vehicle under consideration. In general, 
detection of vehicles passing or being passed is made more robust by first detecting the 
area of the lane(s) adjacent to the lane in which the subject vehicle is moving. Described 

» above is a technique for detecting a solid, broken or double lines. This example will 
assume that a vehicle to be detected is located in the lane 236 to the left of the lane 237 
pf the vehicle under consideration. Initially, it is desirable to detect the line 234 
immediately to the left of the subject vehicle, i.e., the right line of lane 236. This is 
accomplished, as described above, by entering into a search mode until the line is 

D detected, and then remaining in a track mode (240). The left side line 238 of lane 236 is 
generally a known distance d from line 234 at a known difference in orientation. Once 
right side line 234 has been detected, the system enters into a search mode to locate the 
left side line (241). Once the left side line has been acquired, the location of lane 236 is 
known, and controller 42 sets the histogram formation units to search only the area 

5 bounced by lines 234 and 238 by setting XMIN, XMAX, YMIN and YMAX to cover 
only the lane area (242). The axis to be searched is set (244) by controller 42 to be 

parallel to lines 234 and 238. 

Once the search area has been determined, which may occur in as little as 
one frame, controller 42 sets the histogram formation units to detect pixels having 

10 characteristics of a moving vehicle (246). These characteristics are diverse, and depend 
upon numerous factors, such as time of day, ambient lighting, etc. At night, vehicles are 
most easily detected using the high luminance of the vehicle headlights or taillights. 
Taillights may also be detecting using the hue and saturation of the red lights. In daylight 
conditions, motion of the vehicle may be detected by detecting movement in the lane area 

25 (DP), velocity, luminance, movement in a direction parallel to the lane lines, color, etc., 

or combinations of these factors. 

For the present example, it will be assumed that the system is tracking a 
vehicle at night moving in a positive direction, i.e., the vehicle is passing the subject 
vehicle. Controller 42 preferably sets the histogram processing units to detect the hue 
30 and saturation associated with red lights, high luminance values, and velocity in the 
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ZT T" paral,ei ,0 ,he lane fa Hg - 22 *"» the ** »9 Also 

* (although they-axis is not ^ , 0 „, ^ 

cty V values of the ptxels m movement. This velocity value may be verify ^ 
as .he vehtde. The left and right tai „ ighls may „, 

■denunea, the approximate distance between the passing vehicle and the subject v^ 

he de,erm,„at,on tha, the detected objects are a vehicle may be verified by vending that 
•he .wo detected lights are mo ving at the same speed in the same direction Detent 
vehtcles movmg in other directions is accomplished using simiiar tecl^ues 

*« Sh0 ™ in 24 «d 25 the system of the invention makes i, 

perpendicular axes 824. one of them being sensibly perpendicular ,„ said line. ,„ as. a 
-n- « h s to be denned, said curve is segment fat0 sensib|y ^ 
w ere o„ e 0 f , W o perpendicuiar axes 825, 825'. 825- delimiting an area inducing said 
sens,bly hnear portion, is sensibly perpendicular to said linear portion. 

I. will be appreciated .ha. while the invention has been described with 

-pec. to detection of certain types of objects using certain .echoes, , he inventio „ „ a 
g e en lraage ^ , ^ rf ^ ^ ^ ^ 

poss.be measurable characteristics of the pixels, and that any information disceL, 
from he htstograms formed by the invention may be used to detect, .rack, and 
character ,arg*s. More generally, although the present invention has been described 

2 IT * cemin embodimen,s "* variatbns to - «** «■» 

scope of the mventton as described in the following claims. 
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CLAIMS 

1. A process for identifying the orientation of a line in an input 
image, the line comprising a plurality of pixels, the process comprising the steps of: 

forming a histogram of the pixels projected onto a first axis; 

rotating the first axis and forming a histogram of the pixels projected 
onto the rotated first axis until the histogram comprises characteristics indicating that the 
line is most closely perpendicular to the rotated first axis. 

2. The process according to claim 1 wherein the histogram 
characteristics comprise R =NBPTS/RMAX, and wherein the line is determined to be 
most closely perpendicular to the rotated first axis at which R is a minimum. 

3. The process according to claim 1 wherein the first axis is the 
horizontal or vertical axis of the image. 

4. A process of detecting a line on a road from a vehicle-mounted 
camera, the process comprising the steps of: 

acquiring an image of the road, the image comprising a plurality of pixels 
corresponding to the line; 

selecting pixels of the image having characteristics corresponding to 
characteristics of the line; 

forming a histogram of the selected pixels projected onto a first axis; 

analyzing the histogram to identify characteristics indicative of a line; 

rotating the first axis and forming a histogram of the selected pixels 
projected onto the rotated first axis until the histogram comprises characteristics 
indicative of a line. 

5. The process according to claim 4 wherein the step of selecting 
pixels of the image having characteristics corresponding to characteristics of a line 
comprises selecting pixels selected from the group consisting of luminance, hue, 
saturation, direction, DP, CO and velocity. 

6. The process according to ^iaim 4 wherein the step of selecting 
pixels of the image having characteristics corresponding to characteristics of a line 
comprises selecting pixels in a desired area of the image. 
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pixels of ,he ■' aCC ° rdin8 '° ^ 4 Wherei " *• Ste P ° f ■**, 

F«* of ,h ,mage havrng chancres corresponding t0 characteristjcs * 

-p- se tamg pixds „ . ^ ^ of „ e ^ m . ^ ^ • ; : 

8. The process according to claim 4 r^er comprising repeat™ me 

9. The process according t0 cIaim , 

charges comprise R =NBPTS/RMAX> ^ ^ ^ fa . d *» 

mos, c,ose, y perpendicular to ,he reared firs, axis arm. ro,ated firs, axis a, which R is a 



15 



20 



25 



10. The process according to claim 8 wherein the first axis is the 
horizontal or vertical axis of the image. 

andwherein: "' : ^ T" * * ^ » *** *» 

corresn „• * "* ^ ** ° f ^ having charac.eris.ics 
orrespondmg ,„ charac.ens.ics of me fa c „ raprises selecting firM 

*— area of .he image a, a firs, desired orient i„ ft. image for selecting pixels 
assocated wi.h .he firs, fa and selecting p, e ,s in a second desired area of me image * 
asecond des,red ori=„.a,ion in ,he image for se,ec.ing pixels associa.ed with .he second 

the s.e P of forming a histogram of me selected pixels projected 
firs. ax, S comprises forming a firs, histogram of .he selected firs, pixels projected on.o a 
first ax,s and forming a second histogram of ,he selected second pixels projeced onto .he 

nrst axis; 

the step of analyzing me histogram to identity characteristics indicative of 
ate comprises analyzing each of ,te firs, and second histograms to identify 
characteristics indicative of a line; and 
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the step of rotating the first axis comprises rotating the first axis and 
forming first and second histograms of the selected first and second pixels respectively 
projected onto the rotated first axis until each of the first and second histograms 
comprises characteristics indicative of a line. 

12. The process according to claim 11 further comprising repeating 
the step of rotating the first axis and forming first and second histogram until each of the 
first and second histograms comprises characteristics indicative of a line and until at least 
one of the first and second histograms comprises characteristics indicating that the line 
associated with such histogram is most closely perpendicular to the rotated first axis. 

13. The process according to claim 12 wherein the histogram 
characteristics comprise R =NBPTS/RMAX, and wherein the line is determined to be 
most closely perpendicular to the rotated first axis at the rotated first axis at which R is a 
minimum. 

14. The process according to claim 11 wherein the double lines are 
parallel to the other and each line is selected from the group consisting of solid and 
broken lines. 

15. The process according to claim 4 wherein the line is a parallel 
double line and wherein the step of analyzing the histogram to identify characteristics 
indicative of a line comprises analyzing the histogram to identify two peaks characterise 

20 of a parallel double line. 

16. The process according to claim 4 wherein the line is solid or 

broken. 

17. The process according to claim 4 wherein the line is a broken line 

and wherein: . . - 

the step of analyzing the histogram to identify characteristics ind»cative of 

a line comprises time averaging the histogram over a succession of frames of the image. 

18. The process according to claim 4 wherein the line is a broken line 

and wherein: t 



15 



25 



WO 00/11609 



PCT/EP99/004i5 



47 



.he step rfl^ the histogram ,o identify characteristics indicative of 
-mpnses ^ ^ ^ oyer t ^ ^ 

5 and wherein: "" ^^^^^^^ta-.W-B- 

A. step of selecting pixels of the image having, characteristics 
oorrespo„d,„ g to characteristics of , he „„ e compn , es selecting^c 4 ^ 

me, and select^ in . second desfred ^ rf ^ ^ 

assocatedw-thaiecondportionofthehrotalineadjacenttothefirstsection. 

the step forming a histogram of the selected pixels projected onto a first 
» -pnses forming a firs, histogram of the se.ec,ed firs, pixels projected onto Z 

•he step of analyzing the histogram to identify characteristics indicative of 
.hne compnses analyzing ti,e first and second histograms over a succession of frames of 
.Ke ..age to .demify a periodic movemen, of firs, pixels associated with the line from the 
lirst desired area to the second desired area. 

20. The process according to claim 11 further comprising repeating 
e step of rotating tite firs, axis and forming firs, and second histograms until each of 

he firs, and second histograms comprises characteristics indicative of a line and until a. 
eas, one of ,he firs, and second Migrants comprises charac.eris.ics indicating tha, ,he 
ta. assoc,a,ed wi.h such histogram is mos, closely perpendicular ,„ ,he ro,a,ed firs, axis 

21. A process of detecting a lane oh a road from a vehicle-mounted 
camera, ti,e lane being defined by a firs, line on one side .hereof and a second line on ,he 
other s.de thereof, the process comprising the steps of: 

acquiring an image of the road from ,he camera, each of ,he firs, and 
second hnes comprising a plurality of pixels in the image; „ 

destred onen.a.ion in ,he image for selecting pixels associated wi,h d. firs, line, and 
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selecting pixels in a second desired area of the image at a second desired orientation in 
the image for selecting pixels associated with the second line; 

forming a first histogram of the selected first pixels projected onto a first 
axis and forming a second histogram of the selected second pixels projected onto a 
5 second axis; 

analyzing each of the first and second histograms to identify 
characteristics in each histogram indicative of a line; and 

until the. first histogram comprises characteristics indicative of a line, 
rotating the first axis and forming a first histogram of the first pixels projected onto the 
10 rotated first axis, and until the second histogram comprises characteristics indicative of a 
line, rotating the second axis and forming a second histogram of the second pixels 
projected onto the rotated second axis. 

22. The process according to claim 21 further comprising repeating 
the step of rotating the first axis and forming a first histogram and rotating the second 

15 axis and forming a second histogram until each of the first and second histograms 
comprises characteristics indicative of a line and until at least one of the first and second 
histograms comprises characteristics indicating that the line associated with such 
histogram is most closely perpendicular to the rotated first axis. 

23. The process according to claim 22 wherein the histogram 
20 characteristics comprise R =NBPTS/RMAX, and wherein the line is determined to be 

most closely perpendicular to the rotated first axis at the rotated first or second axis at 

which R is a minimum. 

24. A process for detecting a vehicle in an adjacent lane from a 

camera mounted to a subject vehicle, the process comprising the steps of: 

25 acquiring an image of the adjacent lane; 

selecting pixels of the image having characteristics corresponding to 

characteristics of a vehicle; 

forming a histogram of the selected pixels projected onto a first axis; and 

analyzing the histogram to detect characteristics indicative of a vehicle. 
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25. 



A* , k « ^ Pr0C6SS aCC ° rding t0 daim 24 Wherein 1,16 ad iacent lane is 

defined by first and second side lines each of the first anf f a -a v 
«i * . , Muiorine nrst and second side lines comprising a 

plurality of p,xels in the image, wherein: S 

the step of acquiring an image of the adjacent lane comprises- 

0 selecting pixels of the image in a first desired area of the image at a 
first desired orientation in the image for 

^xeis in a second desired area of the image at a second desired orienta « 
the image for selecting pixels associated with the second line; 

first . • J 0 ^ ' ^ ° f S6leCted ** P ke,s P-jected onto a 



10 

second axis; 



) 



BO analyzing each of the ft* and second mognm , 0 idenlify 
charactenst.es m each hologram indicative of a line; and 

iv) until .he first histogram comprises characteristics indicative of a line 
ro a„ g fe ta and fom] , g a fa ^ rf(b ^ , 

otated firs, a*,s, and until the second histogram comprises characteristics indicative of a 
rotattng the seoond axis and forming a second histogram of the second p«els 
projected onto the rotated second axis; and 

the step of selecting pixels of the image having characteristics 
correspond^ to characteristics of a vehicle comprises selecting such pixels in an area 
bounded by the first and second side lines. 

26. The process according to claim 24 wherein the step of selecting 
Ptxels of the image having characteristics corresponding to characteristics of a vehicle 
compnses selecting pixels selected from the group consisting of luminance hue 
saturation, DP, velocity and direction. 

27. The process according to claim 24 wherein the step of selecting 
pub of the image having characteristics corresponding to characteristics of a vehicle 
«mp„ses selecting pixels having a color or luminance characteristic of taillights 
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28. The process according to claim 27 analyzing the histogram to 
detect characteristics indicative of a vehicle comprises analyzing the histogram to 
separately detect each taillight. 

29. The process according to claim 24 wherein the step of selecting 
pixels of the image having characteristics corresponding to characteristics of a vehicle 
comprises selecting pixels having color or luminance characteristics of headlights. 

30. The process according to claim 24 wherein the step of selecting 
pixels of the image having characteristics corresponding to characteristics of a vehicle 
comprises selecting pixels moving in a direction parallel to a direction of the lane. 

31. The process according to claim 25 wherein the step of selecting 
pixels of the image having characteristics corresponding to characteristics of a vehicle 
comprises selecting pixels moving in a direction generally parallel to one of the first or 
second side lines. 

32. The process according to claim 24 wherein the step of analyzing 
the histogram to detect characteristics indicative of a vehicle comprises detecting a 
histogram having NBPTS exceeding a threshold. 

33. An apparatus for identifying the orientation of a line in an input 
image, the line comprising a plurality of pixels, the apparatus comprising: 

a histogram formation unit for forming a histogram of the pixels projected 
onto a first axis; and 

a controller for selectively rotating the first axis, the histogram formation 
unit forming a histogram of the pixels projected onto the rotated first axis, the controller 
analyzing the histogram to determine when the histogram comprises characteristics 
indicating that the line is most closely perpendicular to the rotated first axis. 

34. The apparatus according to claim 33 wherein the histogram 
formation unit comprise a Hough transform unit performing a Hough transform on the 
pixels for enabling rotation of the first axis. 

35. The apparatus according to claim 33 wkjrein the histogram 
formation unit computes R =NBPTS/RMAX, and wherein the controller determines the 
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rotated firs, axis « whidl R i s a minimum to identify the line most closely perpends 
to the rotated first axis. 'penaicuiar 

36 The apparatus according to claim 33 wherein the first axis is the 
horizontal or vertical axis of the image. 

37. An apparatus for detecting a line on a road, which comprises- 
a vehicle-mounted camera acquiring an image of the road, the line 
comprising a plurality of pixels in the image; 
a controller; and 

a histogram formation unit for forming a histogram on pixels having 
selected characteristics on a selected axis, 

.he controller controfflng the histogram formation u „i« t0 se,ec. pixeIs of 
he image having characteristics corresponding to characteristics of a line and to form a 
histogram projected onto a first axis, the controller analyzing the histogram to identify 
character^ indicative of a line, the controller fiirther rotating the firs, axis and 
controlhng the histogram formation unit ,„ forra . histograra projec(ed ^ ^ 

firs, ax, and analyzing the histogram unti, the histogram comprises eharaoenstics 

indicative of a line. 

38. The apparatus according to claim 37 wherein the selected 
characters are selected from the group consisting of luminance, hue, saturation, 

20 direction, DP, CO and velocity. 

39. The apparatus according to claim 37 wherein the histogram 
forma,™ unit comprises an area section, memory for selecting an area of an image for 
which , 0 form a histogram, the controller controlling the histogram formation uni. ,o 
select pixels in a desired area of the image for detecting the line, 

40. The apparatus according to claim 37 wherein the histogram 
formation uni. comprises an area selection memory for selecting an area of an image for 

select pixels in a desired area of the image and to form a histogram at a desired 
M orientation angle for detecting the line. 
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41. The apparatus according to claim 40 wherein the histogram 
formation unit comprise a Hough transform unit performing a Hough transform on the 
pixels for enabling rotation of the first axis for selecting pixels at the desired orientation 
angle. 

42. The apparatus according to claim 37 wherein the controller 
rotates the first axis and the histogram formation unit forms a histogram of the selected 
pixels untU the controller determines that the histogram comprises characteristics 
indicating that the line is most closely perpendicular to the rotated first axis. 

43. The apparatus according to claim 42 wherein the histogram 
formation unit computes R =NBPTS/RMAX, and wherein the controller determines the 
rotated first axis at which R is a minimum to identify the line most closely perpendicular 
to the rotated first axis. 

44. The apparatus according to claim 37 wherein the line is a double 
line and wherein the histogram formation unit comprises an area selection memory for 
selecting an area of an image for which to form a histogram and an angle selection 
memory for selecting an orientation angle for forming a histogram, 

the controller controlling the histogram formation unit for selecting pixels 
of the image having characteristics corresponding to characteristics of a line in a first 
desired area of the image at a first desired orientation in the image for selecting pixels 
associated with a first line of the double line and controlling the histogram formation unit 
for selecting pixels in a second desired area of the image at a second desired orientation 
in the image for selecting pixels associated with the second line, 

the histogram formation unit forming a first histogram of the selected first 
pixels projected onto the first axis and forming a second histogram of the selected second 
pixels projected onto the first axis, the controller analyzing each of the first and second 
histograms to identify characteristics indicative of a line and rotating the first axis until 
each of the first and second histograms comprises characteristics indicative of a line. 

45. The apparatus fording to claim 44 wherein the controller 
rotates the first axis until each of the first and second histograms comprises 
characteristics indicative of a line and until at least one of the first and second histograms 
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comprises characteristic, indicating te the ^ ^ 
closely perpendicular to the rotated first axis. 

46. The apparatus according to Cairn 44 wherein the double lines are 
parallel to the other and each line is selewM «■„ .u 

broken lines. *~ ^ "»*** 0f «*" - 

doubt r f' u ^ aPParatUS aKOniin8 ,0 ^ " WhCTdn *» *» ■» paraUel 
doub,e hr^an wherein Ute confer a^e, ,he Mstogram „ ' 

charactensticofa parallel double line. 

The apparatus according to claim 37 wherein the line is solid or 



broken. 



48. 



49. 



The apparatus according to claim 37 wherein the line is a broken 
line and wherem the controller time averaees th* 
rt , . avera 8 es the htstogram over a succession of frames 

of the image to ,dentify characteristics indicative of a broken line. 

50. The apparatus according to claim 37 wherein the line is a broken 
hne and wherem the controller time analyzes the histogram over a succession of frames 
of the unage to identify a periodic fluctuation in peaks of the histogram indicative of a 
broken line. 

51. The apparatus according to. claim 37 wherein the line is a broken 
to. and wherein the controller controls the histogram formation unit to form a firs, 
urogram of firs, pixe.s in a first desired area of the image associated with a firs, portion 
of the broken line, and to form a second histogram of second pixels in a second desired 
area of the unage associated with a second portion of the broken line adjacent to ,he firs, 
section, each of ,he firs, and second histograms being projected onto the firs, axis rhe 
comroller analyzing ,he firs, and second histograms over a succession of frames of the 
■mage , 0 identify a periodic movement of first pixels associated with ,he line from the 
first desired area to the second desired area. 

52. The apparatus according to claim 44 wherein the controller 
rotates the firs, axis for fonning firs, and secondmistogram until each of the firs, and 
second urograms comprises characteristics indicative of a line and until a. leas, one of 



WO 00/11609 PCT/EP99/00425 

54 

section, each of the first and second histograms being projected onto the first axis, the 
controller analyzing the first and second histograms over a succession of frames of the 
image to identify a periodic movement of first pixels associated with the line from the 
first desired area to the second desired area. 

5 52. The apparatus according to claim 44 wherein the controller 

rotates the first axis for forming first and second histogram until each of the first and 
second histograms comprises characteristics indicative of a line and until at least one of 
the first and second histograms comprises characteristics indicating that the line 
associated with such histogram is most closely perpendicular to the rotated first axis. 

10 53. An apparatus for detecting a lane on a road, the lane being defined 

by a first line on one side thereof and a second line on the other side thereof, the 

apparatus comprising: 

a vehicle-mounted camera for acquiring an image of the road, each of the 

first and second lines comprising a plurality of pixels in the image; 
15 a controller, and 

a histogram formation unit for selecting pixels in an image having 
particular characteristics and for forming a histogram of the selected pixels, 

the controller controlling the histogram formation unit for selecting pixels 
of the image in a first desired area of the image and for forming a histogram of the 
20 selected pixels in the first desired area projected onto a first axis for forming a first 
histogram of pixels associated with the first line, and further controlling the histogram 
formation unit for selecting pixels in a second desired area of the image and for forming a 
second histogram of the selected pixels in the second desired area projected onto a 
second axis for forming a histogram of pixels associated with the second line, 
25 the controller analyzing each of the first and second histograms to identify 

characteristics in each histogram indicative of a line, and 

the controller rotating the first axis until the first histogram comprises 
characteristics Sidicative of a line, and rotating the second axis until the second 
histogram comprises characteristics indicative of a line. 
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54. 



The apparatus according to claim 53 wherein the c „n, „ 

~ *= and the second ^ unta rach rffc ^ _ " J ~ 

"mpnses charactensucs indicative of a line ^ unlil „ _ „ ^ ^ 
*s.ograms compnses characteristic5 indjratjng ^ ^ ^ - 

5 hs,ogran,,s most closely pedicular to .hemmed firs, axis. 

55. The apparatus according , 0 daim 53 ^ ■ 
formation unit computes R =NBPTS/RMAX for each of the first and second histograjns^ 
«. wherem the controller detennines the rotated firs, axis at which R is a minimuT, ■ 
determine when the firs, ,i„e is most closely perpendicular ,o the routed firs, alal 
-0 w ere,„ the confer determines the rotated second a*s a, which R is a 11" 
determine when the second iine is most closely perpendicular to the rotated second axis 

56. An apparatus for detecting a vehicle in an adjacent lane from a 
subject veh,cle, the apparatus comprising: ' ■ 

.5 adjacent iane; 2 " * ^ " " 

forming a histogram of such images; and 

a controller for controlling the histogram formation urn, to select pixels 
having characteristics corresponding to characteristics of a vehicle and for analyzing the 
0 histogram of such pixels to detect characteristics indicative of a vehicle. 

JfJlf ^ WaratUS aCMrdi "S «• «>*» 5« wherein the adjacent lane is 

define by fi M a „ d Mcond side ^ ^ rf fc fe ^ ^ ^ ^ 

plurality of pixels in the image, and wherein: 

the controller consols the histogram formation unit for i) selecting pixels 
of the , mage ,„ a firs, desired area of .he image a. , first desired orientation in the image 
for seating pixels associated with .he firs, side line, and selecting pixels in a second 

desired area of the image a, a second desired orientation in the image for selecting pixels 

assocated with the second side line, and ii) forming a firs, histogram of Selected firs. 

P«e s projected onto a firs, axis and forming a second histogram of the selected second 

pixels projected onto a second axis; 
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the controller analyzes each of the first and second histograms to identify 
characteristics in each histogram indicative of a line; 

until the first histogram comprises characteristics indicative of a line, the 
controller rotates the first axis and controls the histogram formation unit to form a first 
histogram of the first pixels projected onto the rotated first axis, and until the second 
histogram comprises characteristics indicative of a line, the controller rotates the second 
axis and causes the histogram formation unit to form a second histogram of the second 
pixels projected onto the rotated second axis; and 

the controller controlling the histogram formation unit to select pixels of 
the image having characteristics corresponding to characteristics of a vehicle comprises 
in an area bounded by the first and second side lines. 

58. The apparatus according to claim 56 wherein the pixel 
characteristics corresponding to characteristics of a vehicle are selected from the group 
consisting of luminance, hue, saturation, DP, velocity and direction. 
; 59. The apparatus according to claim 56 wherein the controller 

controls the histogram formation unit to select pixels having a color or luminance 

characteristic of taiUights. 

60. The apparatus according to claim 59 wherein the controller 

analyzes the histogram to separately detect each taillight. 
20 6i, The apparatus according to claim 56 wherein the controller 

controls the histogram formation unit to select pixels having color or luminance 
characteristics of headlights. 

62. The apparatus according to claim 56 wherein the controller 
controls the histogram formation unit to select pixels moving in a direction parallel to a 

25 direction of the lane. 

63. The process according to claim 57 wherein the controller controls 
the histogram formation unit to select pixels moving in a direction generally parallel to 

one of the first or second side lines. * 

64. The process according to claim 56 wherein the controller detects a 
30 histogram having NBPTS exceeding a threshold in order to identify a vehicle. 
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65. An apparatus for identifying an object in an input sienaL tfr «v 
comprising pixels in one of a plurality of classes in on ? i T ^ 
input sign, comprising a JLJli^Z^l^ * 
Pixels, the apparatus comprising: * ~" * 

a classifier for each domain, the classifier classuying pixels ^ 
each domain in selected classes within the domain; 

a linear combination unit for each domain, the linear combination unit 

flT ; Si8naI ^ ^ V3,idati0n Acting onell 

ofthe plurality of domains for processing; 

a rotation unit for enabling selection of a histogram formation axis- 

a ta °S™&nnationuni^ 
-put ^a, within tosses selected b y the c , ass if,er within rach dmai „ 
by .he va„da„on signal projected onto the histogram formation axis- mi 

a controller for conning the classifier, linear combination unit, rotation 
un.t, and histogram formation unit for identifying the object. 

66. The apparatus according ,o claim 65 wherein the rotation unit 
performs a Hough transform. rotation unit 

e„„M , ^ aCWrdi " S 10 cla ™ 65 « the rotation unit 

enables selection of a firs, Wstogram formation axis and a second histogram formal 
axis an wherein the histogram formation unit i s capable of fonrnng a L u ™ 

rX 1 hiS '° gram •* ^ * «»*« * «" ' 

projected onto the second histogram formation axis. 

area of th,- ^ 7"' '° claim 65 "herein the object i s in an 
« of the image, and nirther comprising „ area selection unit for selecting an area of 

he image, the .stogram formation unit forming a histogram for pixe.s of the output 
wthin , he selected area of the image, and the controller controlling the la 
selection u„„ for enabling selection of objects in a desired area of an image 

comori/ ■ T' • ***** a " ° bjeCt h » *= object 

compnsing pixels ,„ an area of the image in one of a plurality of classes in one of a 
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plurality of domains, the input signal comprising a succession of frames, each frame 
comprising a succession of pixels, the apparatus comprising: 

a classifier for each domain, the classifier classifying pixels within 
each domain in selected classes within the domain; 

a linear combination unit for each domain, the linear combination unit 
generating a validation signal for the domain, the validation signal selecting one or more 
of the plurality of domains for processing; 

an area selection unit for selecting an area of the image; 

a histogram formation unit for forming a histogram for pixels of the 
output signal within the selected area of the image within the classes selected by the 
classifier within each domain selected by the validation signal projected onto the 
histogram formation axis; and 

a controller for controlling the classifier, linear combination unit, area 
selection unit, and histogram formation unit for identifying the object. 

70. The apparatus according to claim 69 further comprising a rotation 
unit for enabling selection of a histogram formation axis, the histogram formation unit 
forming a histogram for pixels of the output signal projected onto the histogram 
formation axis. 

71 . An apparatus for identifying an object in an input signal, the object 
comprising pixels in an area of the image in one of a plurality of classes in one of a 
plurality of domains, the input signal comprising a succession of frames, each frame 
comprising a succession of pixels, the apparatus comprising: 

a classifier for each domain, the classifier classifying pixels within 
each domain in selected classes within the domain; 

a linear combination unit for each domain, the linear combination unit 
generating a validation signal for the domain, the validation signal selecting one or more 
of the plurality of domains for processing; 

a masking unit for masking an area of ths image to prevent consideration 
of the pixels in the masked area; 
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a histogram formation unit for forming a histogram for pixels of * 
pn each d0 m, n seIected by fc ^ 

formation axis; and msiogram 
«* and h,s.ogram formal p „ it for ^ ^ <■ »"*■» 

10 control signals selected from >he group consisting of ""cludrng 
system, ' ^ "* Se ' eCti " 8 ^ ** <* ^ P-essing 

iv) signals for selecting an area of an image for processing by the in™, 
processing system; and 8 

20 i„ , „• • ° UtPl " ^ ' mage * aem •» *■ "ntroUer 

•ncludmg s,g„a,s resu, t an, from processing the inpu , signa ,s seiected fa. , he " 

consisting of; e*«up 

0 signals containing information on histograms formed in the image 
processing system, and 8 

2j ii) signals containing histograms formed in the image processing system 

selected from n"' ^ '"'^ MC ° rdin8 '° 72 " ta * "» *»*» ^ 

selected from the group conststing of finance, hue, saturation, CO, DP, direction, and J 

informs V 

30 1! ° n ^ f ° med ''" taa8e processing system are selected from ,„! 
group consisting of MIN, MAX, NBPTS, RMAX, POSRMAX. 
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75. An apparatus according to claim 53, wherein the apparatus is built 

in a single chip (MOS). 

76. An interface according to claim 72, wherein the physical link is a 

standard automotive bus. 

77. A process according to claim 1, wherein it is possible to 
dynamically adapt in function of results at least one or more of the following parameters: 
classification, areas, histograms. 

78. Use of an interface according to claim 72 with a physical link 

according to claim 76 and according to claim 77. 

79. A process for identifying the orientation of a line in an input 
image, the line comprising a plurality of pixels, the process comprising the steps of: 

forming histograms of the pixels projected onto multiple axes; 
process until the histograms comprise characteristics indicating that the 
line is most closely perpendicular to one of the multiple axes. 

80. The process according to claim 79, wherein the histograms 
characteristics comprise R = NBPTS/RMAX, and wherein the line is determined to be 
most closely perpendicular to one of the multiple axes at which R is a minimum. 

81. The process according to claim 79, wherein one of the multiple 

axes is the horizontal or vertical axis of the image. 

82. The process according to claim 79, wherein processing is done 

using parallel computation. 

83. A process of detecting a line on a road from a vehicle-mounted 

camera, the process comprising the steps of: 

acquiring an image of the road, the image comprising a plurality of pixels 

corresponding to the line; 

selecting pixels of the image having characteristics corresponding to 

characteristics of the line; 

forming histograms of the selected pixels projected onto multiple axes; 
analysing the histograms to identify characteristics indicative of a line; 
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process until one of the histograms comprises characteristics indicative of 



84. The process according to claim 83 wherein the step of selecting 

pixels of the image having characteristics corresponding to characteristics of a line 

5 compnses selecting pixels selected from the group consisting of luminance, hue 
saturation, direction, DP, CO and velocity. 

85. The process according to claim 83, wherein the step of selecting 
P.xels of the image having characteristics corresponding to characteristics of a line 
compnses selecting pixels in a desired area of the image. 

° 86 - The process ^ding to claim 83, wherein the step of selecting 

Pixels of the image having characteristics corresponding to characteristics of a line 
compnses selecting pixels in a desired area of the image at a desired orientation in the 
image. 

87. The process according to claim 83, further comprising repeating 
5 the step of rotating the multiple axes and forming histograms of the selected pixels until 

one of the histograms comprises characteristics indicting that the line is most closely 
perpendicular to one of the multiple axes. 

88. The process according to claim 87, wherein the histogram 
characteristics comprise R = NPBTS/RMAX, and wherein the line is determined to be 
most closely perpendicular to one of the multiple axes at one of the multiple axes at 
which R is a minimum. 

89. The process according to claim 87, wherein one of the multiple 
axes is the horizontal or vertical axes of the image. 

90. Process according to claim 87, wherein processing is done using 
parallel computation. 

91. A process according to claim 79, comprising the step of 
dynamically adapting in function of the results at least one of the following parameters: 
classification, areas, histograms. 
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CLAIMS 

1. A process for identifying a line in an input image, the 
image comprising a plurality of pixels corresponding to the fine, the 
process including an elementary operation comprising: 
5 - projecting pixels onto a projection axis defined by a 

rotation angle related to a reference axis, as to form a projection 
waveform; 

- analyzing the projection waveform to identify 
characteristics indicative of a line; 

10 characterised in that 

- the projection waveform is an histogram; 

- the image is processed in order to select pixels of the 
image having characteristics corresponding to characteristics of the 
line, the selected pixels being at least from the group consisting of 

15 luminance, hue, saturation, direction, DP, CO and velocity, DP 
identifying a significant variation of a pixel and CO being an 
updated calculated value of time constant; 

- and that N x S elementary operations are done in parallel 
processing with the selected pixels on S =2 sets of N > 3 

20 predetermined projection axis, each set being related to a set 
reference axis {x.y}, the N predetermined projection axis of each 
set being regularly distributed from the set reference axis with a 
stepping angle of 180°/N; 

- and that, if the analysis of the histograms of a set are not 

25 indicative of a line: 

the N predetermined projection axis of said set are rotated 
with a same rotation angle and the elementary operations are done 
until an analysis of the histograms is indicative of a line. 

2. The process according to claim 1 characterised in that the 
30 elementary operations are done in parallel for the S=2 sets, the 

sets being chosen related to {x, y} orthogonal references axis. 

3. The process according to the claim 2 characterised in that 
the references axis are chosen as being the horizontal and vertical 
axis of the image. 
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4. The process according to claim 1 characterised in that the 
elementary operations are done in parallel for the S=2 sets the 
sets being chosen related to {x, y) non orthogonal references axis 

5. The process according to any one of. claims 1 to 4 
5 characterised in that the elementary operations are done in parallel 

for the S=2 sets of N=16 predefined projection axis. 

6. The process according to ope of claims 1 to 5 
characterised in that it further comprises the identification of the 
or.entat.on of the line, the process further comprising the step of 

10 .f the analysis of the histograms of a set are not indicative of 

a ne most closely perpendicular to one of N predetermined 
projection axis of said set: 

_ the N predetermined projection axis of said set are rotated 
with a same rotation angle and the elementary operations are done 
until an analysis of the histograms is indicative of a line most 

closely perpendicular to one of N predetermined projection axis of 

s a io sst. . 

hist™, 7 ' Pr ° CeSS aCC ° rding '° C ' aim 6 cha "^rised in that the 
histogram characteristics comprise- R =NBPTS/RMAX. and in that 
the hne „ determined ,o be most closely perpendicular to the 
predeterm.ned projection axis at which R is a minimum, NBPTS 
being « he number of points in the histogram and RMAX being the 
number of points at the maximum of the histogram 

8. The process according to one of the claim 1 to 7 
characterised in that a camera is mounted on a vehicle and an 

jnput image of the road is acquired in order to identify a line on a 
roaci. 

9. The process according to claim 8 characterised in that the 

Zr 3 TV" 3 br ° ken 3nd that ' he s,e P of -n-lyzlnfl-th. 

histogram to identify characteristics indicative of a line comprises 

tone averaging the histogram over a succession of frames of the 

image. 

10. The process according to claim 8 characterised in that 
he hne on a road is a broken line and that the step of analyzing 

the h.stogram to identify characteristics indicative of a line 
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comprises analyzing the histogram over a succession of frames of 
the image to identify a periodic fluctuation in peaks of the 
histogram indicative of a broken line. 

11 The process according to claim 8 characterised in that 
the line is a broken line and that the step of selecting pixels of the 
image having characteristics corresponding to characteristics of the 
line comprises selecting pixels in a first desired area of the image 
for selecting pixels associated with a first portion of the broken 
line and selecting pixels in a second desired area of the image for 
selecting pixels associated with a second portion of the broken line 
adjacent to the first section; and 

that the step forming histograms of the selected pixels 
projected onto a first set of N predetermined axis comprises 
forming first histograms of the selected first pixels projected onto a 
first set of N predetermined axis and forming second histograms of 
the selected second pixels projected onto the first set of N 

predetermined axis; and f 
that the step of analyzing the histogram to identify 
characteristics indicative of a line comprises analyzing the first and 
second histograms over a succession of frames of the .mage , to 
identify a periodic movement of first pixels associated with the line 
from the first desired area to the second desired area. 

12 The process according to claim 8 characterised in that 
the line is a broken line and that the step of selecting pixels of the 
Image having characteristics corresponding to characterist.es of the 
line comprises selecting pixels in a first desired area of the .mage 
'for selecting pixels associated with a first portion of the broken 
line and selecting pixels in a second desired area of the .mage for 
selecting pixels associated with a second portion of the broken line 
30 adjacent to the first section; and 

that the step forming histograms of the selected pixels 
projected onto sets of N predetermined axis comprises forming. first 
histograms of the selected first pixels projected onto a first set of N 
predetermined axis and forming second histograms of the selected 
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second pixels projected onto a second set of N predetermined axis- 
and 



that the step of analyzing the histograms to identify 
characteristics indicative of a line comprises analyzing the 
5 histograms from first and second sets over a succession of frames 
of the image to identify a periodic movement of first pixels 
associated with the line from the first desired area to the second 
desired area. • , 

13. The process according to any one of claims 8 to 12 
10 characterised in that the line is a parallel double line and that the 

step of analyzing the histogram to identify characteristics indicative 
of a line comprises analyzing the histogram to identify two peaks 
characteristic of a parallel double line. 

14. The process according to any one of claims 8 to 12 
15 characterised in that a camera is mounted on a vehicle and an 

mput image of the road is acquired in order to identify a line on a 
road which is a double line and that the step of selecting pixels of 
the ,mage having characteristics corresponding to characteristics of 
the l.ne comprises selecting pixels in a first desired area of the 
20 ,mage at a first desired orientation in the image for selecting pixels 
associated with the first line, and selecting pixels in a second 
des.red area of the image at a second desired orientation in the 
image for selecting pixels associated with the second line; and 

that the step of forming histograms of the selected pixels 
projected onto sets of N predetermined axis comprises forming first 
, h.stograms of the selected first pixels projected onto a first set of N 
predetermined axis and forming second histograms of the selected 
second pixels projected onto the first set of predetermined axis- 
and 1 

that the step of analyzing the histograms to identify 
characteristics indicative of a line comprises analyzing each of the 
first and second histograms to identify characteristics indicative of 
a line; and 

that if the analysis of the histograms of the first set are not 
35 indicative of a line: 
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the N predetermined projection axis of said set are rotated 
with a same rotation angle and the elementary operations are done 
until an analysis of the histograms is indicative of a line. 

15. The process according to any one of claims 8 to 12 
5 characterised in that a camera is mounted on a vehicle and an 

input image of the road is acquired in. order to identify a line on a 
road which is a double line and that the step of selecting pixels of 
the image having characteristics corresponding to characteristics of 
the line comprises selecting pixels in a first desired area of the 

10 image at a first desired orientation in the image for selecting pixels 
associated with the first line, and selecting pixels in a second 
desired area of the image at a second desired orientation in the 
image for selecting pixels associated with the second line; and 

that the step of forming histograms of the selected pixels 

15 projected onto sets of N predetermined axis comprises forming first 
histograms of the selected first pixels projected onto a first set of N 
predetermined axis and forming second histograms of the selected 
second pixels projected onto a second set of predetermined axis; 
and 

20 that the step of analyzing the histograms to identify 

characteristics indicative'of a line comprises analyzing histograms 
of first and second sets to identify characteristics indicative of a 
line; and 

that if the analysis of the histograms of a set are not 
25 indicative of a line: 

the N predetermined projection axis of said set are rotated 
with a same rotation angle and the elementary operations are done 
until an analysis of the histograms is indicative of a line. 

16. The process according to any one of claims 8 to 15 
30 characterised in that a camera is mounted on a vehicle and an 

input image of the road is acquired in order to detect a lane on a 
road from the vehicle-mounted camera, the lane being defined by a 
first line on one side thereof and a second line on the other side 
thereof, the process comprising the steps of: 
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acquiring an image of the road from the camera, each of the 
first and. second side lines comprising a plurality of pixels in the 
image; 

selecting pixels of the image in a first desired area of the 
5 image at a first desired orientation in the image for selecting pixels 
associated with the first side line, and selecting pixels in a second 
desired area of the image at a second desired orientation in the 
image for selecting pixels associated with the second side line; 

identifying the first side line and the second side line. 
10 17. The process according to claim 16 characterised in that 

a camera is mounted on a vehicle and an input image of the road is 
'acquired in order to detect a vehicle in an adjacent lane from a 
camera mounted to a subject vehicle, the adjacent lane being 
defined by a first line on one side thereof and a second line on the 
15 other side thereof, the process comprising the steps of: 

acquiring an image of the road from the camera, each of the 
first and second side lines comprising a plurality of pixels in'the 
image; . 

detecting an adjacent lane by identifying the first side line 
20 and the second side line of said adjacent lane; 

selecting pixels of the image having characteristics 
corresponding to characteristics of a vehicle; 

the step of selecting pixels of the image having 
characteristics corresponding to characteristics of a vehicle 
comprises selecting such pixels in an area bounded by the first and 
, second side lines; 

forming a histogram of the selected pixels projected onto a 
first set of predetermined axis; and 

analyzing the histogram to detect characteristics indicative 
30 of a vehicle; 

18. The process according to claim 17 characterised in that 
the step of analyzing the histogram to detect characteristics 
indicative, of a vehicle comprises detecting a histogram having 
NBPTS exceeding a threshold, NBPTS being the number of points 
35 in the histogram. 
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19 The process according to claim 17 or 18 characterised in 
that J step of selecting pixels o, the image having <*aracter,s ,cs 
"ponding to characteristics of a vehicle >«?%-^™ 
oixels having a color or luminance characteristic of taillights 
P 20 L process according to claim 19 analyzing the 
histogram to detect characteristics indicative o, a vehicU , comprises 
Lalvzina the histogram to separately detect each tailhght. 

V 2 The process according to Calm 17 or 18 characterised ,n 
that the step of selecting pixels of the image having characters 
^ ending «o characteristics of a -Mole comprise.se ecting 

Lvino a color or luminance characteristic of headlights. 
P ' X8lSh /; ,n T r Is according to claim 21 analyzing the 
Mstogram to detect characteristics indicative of a '^^'^ 
analyzing the histogram to separately detect each headlight. 

23 The process according to one of claims 17 to 22 
■ a \ that the step of selecting pixels of the image 

direction of the lane. . 17 to . 2 3 

24 The process according to one of cla.ms 17 to 

• h n that the step of selecting pixels of the .mage 

to one of the first or second side lines. „,,„,, 
• 25 The process according to one of claims 17 to 22 

narameters: classification, areas, histograms. 

26 An apparatus for identifying the orientation of a line n 
0 an input' imagine line comprising a plurality of pixels, said 

'^^o^ Pronto a proton axis defined by a 
rotation^* rented to a reference axis, as to form a promotion 
waveform; 
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- -analyzing the projection waveform to identify 
characteristics indicative of a line; " 

characterised in that the projection waveform is an 
h.stogram and that it comprises means for executing the process of 
ny one of previous Cairns, said means being a, leas, a 7m g e 
processing system interfaced with a controller: 



-the ,mage processing system comprising histogram 
forma.,on umts and rotation units for S - 2 sets of N >3 

10 w T".'- Pr0ieCti0 " axis ' said histogram formation units 
10 formmg h.stograms of the pixels projected on the predetermined 
project™ axis of sets, and said rotation units rotating by a Zll 
ro.at.on angle the predetermined projection axis of sets 

th. hiJ COn, , r °" er ana ' y2ing ,he hist °9 ra ™ ^ determine when 
he h,stograms of a set comprises characteristics indicating that the 

of a se ; r^r^ rr rota,e the n — - 



thatthfrn/I 16 aPParatUS aC ° 0rding t0 C ' aim 26 oharacterised in 
1 Houch f COmPriSeS 3 H ° U9h tranS1om «nrt- Performing 
^tlirTar °" PiXe ' S -——ion o, J 

in tha't 8 ' ZVSZT aC0 ° rding '° C ' aim 26 ° r 27 oharacterised 
'" that each h.stogram formation unit comprises an area 
seleCon memory for selecting en area of an image for which " 
.form a histogram, the controller controNing the histogram formation 
um, to seiec, pixe.s in a desired area of the image for detect "he 

ch,^., 2 9 '- ™ eappara,us accordi "9 to any one of claims 26 to 28 
characterised ,n that the interface between the image processing 
system and the controller comprises: Processing 

input signals from the controller to the image processina 
system including control signals seiected from the group'consislg 
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n signals for selecting domains for processing by the image 
processing system, said domains being selected from the group 
clistU of luminance, hue, saturation, CO, DP, directs, and 

Vel ° C,t ii) signals for selecting classes of pixels within each domain 
for processing by the image processing system, 

iii) signals for selecting rotation angle of sets for 
formation of histograms prelected on the predefined axes of the 

^ >) signals for selecting an area of an image for processing 
by the image processing system; and _ _ 

output signals from the image processing system to the 
controller including signals resultant from processing the mput 
signals selected from the group consisting of: 

i) signals containing information on histograms formed in the 

imaqe processing system, and 

ii) signals containing histograms formed in the .mage 

processing system. . . 

30 The apparatus according to claim 29 characterised ,n 
20 that the' signals containing information on histograms formed m 
he image processing system are selected from the group 
consisting of M.N. MAX, NBPTS, RMAX, POSRMAX, MIN b..ng the 
minimum of an histogram, MAX being the maximum of an 
histogram, POSRMAX being the position of the maximum of an 

25 histogram. . nc . 

31 The apparatus according to any one of claims 26 to 

>30, characterised in that the apparatus is built in a single ch.p 

(M0S) 32 The apparatus according to any one of claims 26 to 31, 
30 characterised in that it further comprises a physical link wh.ch is a 

standard automotive bus. 

33 A device for identifying an object in an input signal, the 
object comprising pixels in one of a plurality of classes in one of a 
plurality of domains, the input signal comprising a succession of 
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frames, each frame comprising a succession of pixels, the 
apparatus comprising: 

a classifier (25b) for each domain, the classifier 
ciassrfying pixels within each domain in selected classes within 
the domain; 

a linear combination unit for each domain, the linear 
combination unit generating a validation signal for the domain the 
val.dation signal selecting one or more of the plurality of domains 
for processing; 

rotation units for rotating S=2 sets of N >3 predetermined 
projection axis; 

histogram formation units for forming histograms for 
pixels of the output signal within the classes selected by the 
classifier within each domain selected by the validation signal 
projected onto sets of predetermined projection axis; and 

a controller for controlling the classifier, linear combination 
unit, rotation unit, and histogram formation unit for identifying, the 
object. 

34. The device according to claim 33 wherein the rotation 
unit performs a Hough transform. 

35. The device according to claim 33 or 34 wherein the 
rotation units enable the rotation of a first set of predetermined 
axis and of a second set of predetermined axis, and wherein the 
h.stogram formation units are capable of forming first histograms 
projected onto the first set of predetermined axis, and of forming 

^second histograms projected onto the second set of predetermined 
axis. 



36. The device according to any one of claims 33 to 35 
characterised in that the object is in an area of the image 
and the device further comprises an area selection unit for 
selecting an area of the image, the histogram formation units 
forming histograms for pixels of the input signal within the 
selected area of the image, and the controller further controls the 
area selection unit for enabling selection of objects in a desired 
area of an image. 
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37. The device according to any one of claims 33 to 36 
characterised in that the object is in an area of the image, and the 
device further comprises a masking unit for masking an area of the 
image to prevent consideration of the pixels in the masked area, 
and the controller further controls the masking unit for identifying 
the object. 
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